745 Comments
Comment deleted
Expand full comment

I agree, but just so you know, it saves your responses even if you don't finish. For me, it took about thirty minutes including breaks for eyestrain.

Expand full comment
Comment deleted
Expand full comment
Comment deleted
Expand full comment

I read it twice and was still totally confused.

Expand full comment

I believe you can still skip the questions, even though the radio buttons may not let you unselect after you picked an aswer at random

Expand full comment

I'm half Japanese, quarter French, and quarter German. How should I answer the race question?

Expand full comment

Also, the profession question. I don't work or study anything at all.

Expand full comment

"Have you completed your formal education?" I'm a college dropout. I'd like to go back someday, but have no current plans.

Expand full comment

I'd say "no" to that one, I guess. I'm a drop-out with no desire/plan to back, so I said "yes."

Expand full comment

It feels weird to answer that way because it makes basically no actual difference either way. It's just something I tell myself.

Expand full comment

Ah, but are not the stories we tell ourselves amongst those most important?

Expand full comment

No, I don't think so. Not if they have no bearing on reality.

Expand full comment

The standard way to do this is "mark all that apply", disappointing that Scott's using outdated methodology.

Expand full comment

"Mark all that apply" is useful if you want humans to visually inspect the results, but makes things difficult if you want computers to analyze the results. With answers that are either yes/no, or a number, you can do a regression, or logistic regression, factor analysis, PCA, or lots of other powerful numerical algorithms. If you've got a bunch of boxes that can be checked or not checked, with no known relationship between those boxes, what are you going to do with it?

Expand full comment

Personally, I barely managed to resist the temptation to answer "Other: 100 yard dash"

Expand full comment

If you want to give those answers in excrutiating, painful detail, you can totally answer the optional demographic questions on the COVID-19 Impact survey!

Expand full comment

There being other questions on other surveys doesn't change the fact that there are these questions to answer on this survey.

Expand full comment

What are your feelings on the Alsace-Lorraine question? 😉

Expand full comment

My feeling is that the Schleswig-Holstein Question is the more interesting one.

Expand full comment

Was that on a different survey? I don't recall it.

Expand full comment

You said you are quarter french/quarter German, she made a joke about a question that divides these nationalities.

Expand full comment

Ah. Well, just goes to show how little connection I have with my heritage...

Expand full comment

White

Expand full comment

It's a bit late for that, for a more modern version "What are your feelings about the EU?" could get you some nice answers.

Expand full comment

It's never too late for this type of thing… and historically Taunton is a part of Minehead, I'd have you know!

Expand full comment

Personally, I skipped that answer. To me as a German, anything about race is super scary and weird, because it immediately evokes my countries history.

TLDR: Just skip it, I would say it's a weird US thing

Expand full comment

You shouldn't skip it just because it seems weird. Rather, all the more reason to investigate.

Expand full comment

Let me be more clear: I'm disgusted by the way Americans still file data on and talk about race like it's the 1940s. If you want to talk ethnicity or social standing that's fine, but everything else should not be in data like this. If the German federal state would ask for race data in, say, passports, the public outcry would be massive.

"more reason to investigate" - no, pretty uncool since nazi times.

I am aware that critical race theory exists, and how it's important in the US etc, but I'd rather not import the US notions of race into my own country nor thinking.

Expand full comment

Huh, that's a really interesting cultural difference. (I'm from the US.) If you don't mind me asking you to elaborate: Do most Germans consider themselves "colorblind" / think racism is a thing of the past? Or do most people generally feel like racism is still an issue in modern German society, but that explicitly categorizing race is part of the problem?

Expand full comment

Should have noticed this when these were first posted, but weaknesses in the demographic survey:

- No multiracial

- No Native America or Pacific Islander

- Hispanic is not a race, though I selected it (I am multiracial White and Native American)

- I have no clue whether people within two generations of my family have been diagnosed with specific mental illnesses and I'm not sure how widely that type of thing gets discussed outside of immediate family

- Was the above intended to only include direct ancestors? Because I took it to include siblings, cousins, aunts, and uncles

- You gave us the ability to say we're not interested in politics and not registered with a party, but did not give the ability to indicate we don't feel any particular allegiance to any of the listed named ideologies, either

Expand full comment

I don't understand much about how the US thinks about their races so forgive me for my ignorance but if "Hispanic" is not a race where do you put people coming from Latin/Hispanic America?

Expand full comment

By their race. White, black, mixed...

Expand full comment

Those are skin colors, not races.

Expand full comment

As a US person I think the Hispanic/non-Hispanic scale is gesturing at indigenous ancestry from different regions, and also at language, but then also at situations of general "otherness." The scale doesn't make sense to many people in the US either. Spanish and British colonialism worked out differently. British did not create the "mixed race Indian and something else" classes so that even if multiracial people with Indian ancestry existed they were subsumed into white or black. Spanish colonialism did create the category of "mixed race Indian and something else" and so the categories don't map neatly from one region to another.

Expand full comment

My impression was that "white" meant "mostly european ancestry", and thus hispanic was useful because it meant "mostly central and south American ancestry", even if those tend to be more mixed than Europe. The thing is, I think two things are mixed in the USA definition of race: the genetic origin of people, and how people are categorized into groups by other people based on their appearance, culture, the way they act. The first part is the "medical" part of the definition of race, and is useful in cases like sickle cell disease. The second part is the "sociological" definition of race, and is useful in cases of discrimination, racism, and studying society in general. The way both are mixed strikes me as unhelpful.

Expand full comment

As Marginalia said, the "Hispanic" category is cultural, an artifact of the Spanish empire creating a distinct mestizo identity and subsequent countries, most notably Mexico, explicitly embracing mestizo identity as their national identity. This is quite different from the one-drop rule of the United States, but just like the United States, the people populating other American countries, including Latin American countries, were either natives or they came from somewhere else, mostly colonizers from Europe and slaves from Africa, and their races are the same as the races of the people in the United States. The biggest difference is there are far more people of partial or majority native descent.

Expand full comment

Most official documents (employment applications and the like) have one question for "race" with a range of races, and a separate question for "ethnicity", with the two options of "Hispanic/Latino" or "Not Hispanic/Latino". As detailed in this presentation on the 2020 census: https://www2.census.gov/about/training-workshops/2020/2020-02-19-pop-presentation.pdf this dates back to the 1970 census, when the Office of Management and Budget defined "ethnicity" as a technical term referring solely to whether or not someone is Hispanic/Latino. Apparently, research has shown that integrating Hispanic-or-not into the "race" question is better design....but the 1997 OMB standards require that the question be asked separately. All the employment and university applications I've filled out follow this pattern, likely for similar legal reasons.

Expand full comment

What's with all the gender Ideology stuff in the 0: General Demographic Information survey?

At that I'm out ⛔

It looks more like an idealogical purity test than a demographic survey.

Sex Not Gender ♂️♀️

Expand full comment
Comment deleted
Expand full comment

I ignored my true beliefs on the matter and ticked the cis box, but my vast amounts of privilege makes this kind of lie very easy for me to bear.

Expand full comment
author

That's copied from my last survey demographic section, but I had it in there for a few reasons:

1. One of the things I used the survey for was to investigate what things are due to biological sex vs. self-identified gender. Without distinguishing between those, that would be impossible. I'm not going to prevent myself from investigating important questions just to avoid offending people. See https://slatestarcodex.com/2017/06/28/why-are-transgender-people-immune-to-optical-illusions/ for an example of some research I did with those questions.

2. Without that distinction, transgender readers might be confused about what answers to give, or people interpreting the survey might be confused about what their answers meant. Regardless of your ideology it's important to make sure you're communicating clearly with your subjects and that their answers mean what you think they mean.

3. Past research has suggested that about 3-4% of ACX readers are transgender, which is significantly overrepresented compared to the general population. I'm interested in tracking that and investigating why.

Expand full comment
Comment deleted
Expand full comment

Interesting hypothesis! Let's see how the survey data pans out--I think there was a question in one of em about autism iirc?

Expand full comment

Scott, I appreciate your taking the time to respond to my rather brief message.

⛔ I'm not in the least offended but I do object to the normalisation of a particular socio-political / idealogical framework as the normative view of human categories. That I most certainly do object to. ⛔

If the purpose of the questionaire was to determin the embeddedness of a particular Ideology in the readership then fine, we'd all know the purpose and could continue on that basis but this does not seem to be the case.

If that were the case then we should have a control group using normative categories to compare and contrast against and we don't have that, so again I question the appropriateness of it's inclusion.

And so to restate my position, and with regret, I do not feel I can contribute to a ideo politically compromised questionaire without providing tacit approval to its ideo political contents.

Expand full comment
author

"If that were the case then we should have a control group using normative categories to compare and contrast against and we don't have that, so again I question the appropriateness of it's inclusion."

I'm confused - what would that mean?

Suppose the readership is 100 cis men, 100 cis women, 10 trans men, and 10 trans women.

You can already tell that if you just ask "what is your gender?" without allowing anything about trans as an option, 110 of them will say male and 110 will say female.

Can you explain more how a control group would work?

Expand full comment

Once again thanks for replying.

My objection ⛔ is to the highly partisan language and idealogical basis of the foundational questions in this questionnaire.

You've chosen language only supported by ~8% of the american population, at most. And much less popular is other English speaking countries around the world.

https://hiddentribes.us/profiles/

Now a questionnaire on the relative support of Gender Ideaology would be a very worthwile thing to do in it's own right but mixing it's highly contentious and partisan language and ideas in has I believe biased this questionaire at it's outset and my participation in it would be to support these highly divisive idealogical ideas, which I will not support. ⛔

I'm sure such ideopolitical framing would help ward off any future New York Times hit pieces though 😕

Expand full comment

Again agree. Control of language is power of itself. See most of Orwell's work for further detail

Expand full comment

But on that particular contentious topic, I despair at finding ANY set of terminology that's "neutral"; EVERY set of words to describe anything related to sex or gender has become politically loaded and subject to endless fights, as seen in the edit history and discussion page of any Wikipedia article on the topic. (Myself, I don't like the "Assigned * at birth" construction... what, is there a Sorting Hat making the assigment? Biological sex is *observed* at birth.)

Expand full comment

I would be absolutely shocked if only 8% of the US population was supporting the idea of sex vs gender. You link specifies Progressive Activists, but i am most definitely not one and i support the idea.

Expand full comment

Yep. This commenter is clearly a crank

Expand full comment

I somewhat disagree with Gerd, but he/she/it/etc. has a point: it might be a good idea to include fallback survey options such as "I disagree with the very notion of gender as separate from sex", or something of the sort.

Expand full comment

Edit: as I'd mentioned above, it might be good to include some options such as "I'm just a man/just a woman", in addition to "cis/trans".

Expand full comment

Great idea. The people who select "just an X" are different from those who select "cis/trans."

Expand full comment

Good suggestion.

Expand full comment

You are presuming a normative framework, but demography is a descriptive framework.

If you believe that trans people don't exist, or that they shouldn't list, or that they are deluded perverts, or whatever, that is your normative perspective.

But a descriptive approach acknowledges that there are a whole bunch of people who believe gender and birth sex do not align in their case. Failing to account for that perspective makes your data worse. If you just ask "man/woman" when you really want to know birth sex, you get junk data. Even if a researcher wanted to use your normative categories, they couldn't trust their responses to properly disambiguate the data.

I am trying very hard to be charitable, but even if I agreed with your ideological position (and to be clear, I don't), I'm not sure how avoiding the question would allow you to get at the information you want.

Expand full comment

Surely, even from the point of view that transsexuals are delusional, it is useful demographic information to know whether a given subject suffers from said delusion?

I don't see anything in the survey that assumes self-identified gender is correct - it doesn't ask "what gender *are* you", but "with what gender do you primarily identify".

Expand full comment

I think this is like objecting to research programs whose goal is to study some topic that isn't yet thoroughly understood.

Expand full comment

I think you are close to a very important insight which can allow you to better empathize with people around you. The thing you've experinced is exactly how being offended feels from inside. This rightful fury, after visualizing terrible consequences if the things would continue to go this way, this feeling of menace, of violation of someones freedoms and liberties, of being enforced some categories and narratives which you didn't consent to - is exactly the way one of these ~8% of the american population would feel reading your reply and for mostly the same reasons!

We may want to distinguish between this cluster of feelings from bitterness or resentment. Something a children may feel when their wish isn't immediately satisfied. Is it what you've meant when saying that you are not in the least offended - that you are not like this children, that your concerns are much more valid? I can agree there can be a meaningful distinction and if so, then virtue of precision motivates us to try to reshape the categories to grasp this distinction. But by exactly the same logic there is a meaningful distinction between sex and gender categories and we ougth to grasp it in order to be more precise.

It's sad that these topics are politicised. But we don't have to accept this narrative of politization. We can just try to talk about categories as precise as we can trying to figure correlationsand causality between them. Imagine how annoying and counterproductive it would be if question of not being offended was heavily politicised. If people would immediately and with straight face start to tell you that you are a deluded liar for claiming not to be offended when you clearly are, that words have meanings and you are just trying to smuggle your political agenda into the conversation.

Expand full comment

Many readers of this blog and takers of this survey, myself included to some extent, would see a *lack* of acknowledgement of the existence/validity of transgender identities as an objectionable ideological framework.

Expand full comment

Maybe you need a “This survey is shit button—I’m out” button to track how many people see things as starkly as Gerd that you just aren’t tracking. Might be useful to know.

Then again it’s also what the comments section is for to some extent.

Expand full comment

I'm a white hetero married Republican Mormon, and I'm perfectly fine with answering that question. I'm curious about the data.

Expand full comment

Also just probably outed myself in the survey data, huh. But that's OK. I'm not too worried about my answers.

Expand full comment

Yep, there can't be too many of those around here, even considering the likely positve correlation of those traits.

Expand full comment

He definitely didn't out himself.

Expand full comment

I agree that from one viewpoint, it's annoying. But there are many people out there who are very much of a different viewpoint, so for the sake of taking a survey, if someone wants to identify as non-binary trans masc panromantic soft femme, let 'em and let's get on with it. I'm not gonna fight with someone else over their weirdness if they don't fight with me over mine.

Expand full comment

Same. I am not a cis woman. I am a woman. The "cis" label implies a whole bunch of baggage that I am not going to hold.

Expand full comment

This doesn't make sense to me, because the "cis" label implies the explict *absence* of any such baggage. All that weird gender identity stuff I don't understand or relate to? Doesn't apply to me. Won't hold it. Therefore, cis. The fact that this is relevant to *other* people's identity doesn't make the label that distinguishes me as "not one of them" an indictment, or imply that I endorse the ideology or whatever.

It's like, at first I live in an all-white neighbourhood and I don't even know the word 'Caucasian'. Then a black family moves in and they say "We're African-American, you're Caucasian." If they claim that they're deserving of better treatment because of this distinction, I might disagree. But I don't fight against being labeled as different when we *are* different.

Expand full comment

There's another way of looking at this: the "cis" label implies that there can be something else than a cis woman. I think that's what C MN is referring to.

Expand full comment

It might socially imply that, but it doesn't logically imply it.

BUT ALSO: If God (or a sufficiently advanced alien) 'changed' your sex in every possible meaning of the word 'sex' that you personally subscribe to, would that not make you the opposite sex by your own definition?

Expand full comment

I'm not sure what you mean by "logically implying" here, but surely if you say "cis women", you're using "cis" as a modifier for "women" and thus saying that there are one or multiple "* women". If not you wouldn't use "cis" before women?

> If God (or a sufficiently advanced alien) 'changed' your sex in every possible meaning of the word 'sex' that you personally subscribe to, would that not make you the opposite sex by your own definition?

I think that would, but I don't see what you're getting at.

Expand full comment

1. Saying 'large elephant' doesn't imply that small elephants exist.

2. If god made man into a woman, they would not be a cis woman (cis meaning assigned at birth). You seemed to be saying it is not logically possible for there to be such a thing as a non-cis woman, but this is an example where you agree there could be.

If God made 1% of all men into women, wouldn't 'cis' become a useful and meaningful term that is fine to include on surveys where relevant?

Expand full comment

I agree completely -- the term "cis" does impose a certain worldview on the reader. I have no problem with it myself, but I can see how others might. A better survey design might look something like this:

"I identify as:

( ) Cis Man

( ) Cis Woman

( ) Trans Man

( ) Trans Woman

( ) Simply a Man

( ) Simply a Woman

( ) Non-binary

( ) Gender-fluid

( ) Other

Expand full comment

That's a really good way to put it.

Expand full comment

Excellent solution. I think it is important to allow the option of conscientiously objecting to the implications of politically loaded and often intrinsically normative terminology being introduced under the guise of being merely descriptive

Expand full comment

Yes, as it stood, I just wrote it under "other." I'm just a body! My experience of my own brain isn't gendered (unsure how I'd even know if it was).

Expand full comment

Better yet to simply ask in questionnaire #0

Male ()

Female ()

Prefer not to say ()

and put these more ideo political questions within their relevant Gender Identity sub questionnaire #17 for those that subscribe to those particular beliefs.

Everyone would have been catered to and nobody left out.

Expand full comment

Agreed. I've been taught survey writing at the college level and that's how I would have handled it. Keep biological sex as its own question, and then for the people who believe they have a "gender identity" or whatever, they can answer question(s) about that.

Expand full comment

But then you would get ambiguity--people who do believe that sex and gender are separate would not be clear whether they should put their biological sex or identified gender if there was only one question with only those categories.

Expand full comment

A more fitting analogy - from the perspective of the critics here - would be if a family claiming to be aliens moved into your neighborhood, and thenceforth you had to identify as a 'terrestrial' in surveys, which would implicitly affirm the idea that some of their neighbors are actually extraterrestrials, which might be irksome.

Expand full comment

I see your point, I'd also be irked if surveys started asking me if I identified as a flat-earther in the same way they asked me about my gender, if only because it suggests that the Overton window has shifted towards flat-earth in a way that's unacceptable to me. But then refusing to take the survey, or demanding that they *not* ask anyone whether they're a flat-earther, just seems like sticking my head in the sand. Flat-earthers exist, no matter how much I wish they don't. Taking that out on survey-takers is actively counterproductive.

Expand full comment

As I said above, a better survey design would have questions that capture this view; something along the lines of, "I'm just a normal person and I don't think aliens exist", as opposed to, "I'm terrestrial, implying that I'm not an alien, even though that is technically correct".

Expand full comment

I don't see how it implies that. I am definitely a terrestrial bipod primate and saying that doesn't imply any of the people in my neighborhood I'm reasonably sure are also human might actually be extraterrestrials. If it was the case that some small minority believed themselves to be extraterrestrials and this had interesting social and policy implications, though, it would be useful to capture that information in demographic surveys.

Expand full comment

Bad analogy.

You're talking about a disagreement in empirical facts - where they were born.

There's no disagreement on empirical facts in the trans debate (or at least, not among educated members of the debate on both sides.)

Objecting to someone saying empirical facts which aren't true is different from objecting to someone changing semantic definitions of words.

Expand full comment

The "what do you identify as?" language seems to make both this and the original question about as fact-based and non-controversial as they can possibly be. Trans-women/men, defined as "People who were born in a male/female body and currently say about themselves that they are actually women/men" are definitely a group who exist - a question about that is entirely appropriate. In the analogy "do you identify as terrestrial or alien?" is a perfectly valid question. The fact that you may think "those people who identify as aliens are complete loons who are just as terrestrial as me" is not captured by your replying "I identify as terrestrial" - but it doesn't have to be.

Expand full comment

Funny, because I think it's the very framing of "what do you identify as" that is problematic. By asking what one "identifies" with, the implication is that in so "identifying," one accepts, likes or is content with or at least tolerates, and/or finds some sense of meaning in said identification. Which is not at all the case for me, with respect to my biological femaleness -- i.e. I am and have always been a female, but I certainly had no choice about that, and I do NOT "identify" with "being a woman" or "womanhood" (not that I even know what "womanhood" is supposed to mean).

No one asks whether I "identify" with/as my actual demographic categories of height, age, ethnicity, citizenship, weight, eye color, or anything else. Those things are just facts, ones that you can't control and didn't have any choice about. In one COULD identify as their preferred weight category, surely there would be a whole lot of overweight people who "identify" as thin. Surely plenty of old people "identify with" being young.

I selected "cis woman" because I am heterosexual female and I'm not trans or nonbinary or whatever all the other categories are. But I certainly don't "identify" with or as a "woman" if that means some type of socially-prescribed but amorphously prescribed gender role or necessarily LIKE or feel comfortable as a woman. I just am one and it would never occur to me that there was anything I could do about it or that anyone would care how I felt about it, and the same way that there is nothing I can do about my height or age, and therefore how much I "identify" with those things is completely irrelevant.

I didn't have a problem with the survey question, and I think it's necessary to include a question that clarifies which respondents are trans, but I can see that there is a problem with "cis gender" in that presumes that a person believes in gender identity at all, or identifies as anything.

Expand full comment

It *is* interesting, because I think I probably have a very similar attitude towards womanhood/femininity to what you are describing - that is, yeah, I have a female body therefore I'm a woman (*shrug*). I'm deeply suspicious of the very concept of "feeling like a woman/man" - that is, I believe people when they say they experience life this way, I don't experience it myself, I don't really understand how you can be so sure that you "feel like" ... some any disparate group of people with their own individual feelings about things.

But I take the whole "identify as" question much more operationally - if someone says 'are you a man or a woman?' what do I say? And, yeah, I am saying "woman" because I'm not taking gender-orthodoxy seriously (which would presumably prescribe that I should say "non-binary") and solely working on the basis of physical body. But they didn't ask that!

Expand full comment

It occurs to me that Social Justice makes it seem plausible to be very concerned about how one is perceived demographically, even if one is generally in disagreement with social justice.

Expand full comment

It is necessary to talk about trans issues, and impossible to talk about them without making the cis / trans distinction. It's okay to redefine our terms within a specific discussion. Words were not handed down by God.

Expand full comment

It doesn't, though.

Like, if I said 'anyone with a nose that extends between 2 and 5 cm from their lip is a flim', and that describes you, then calling you a flim doesn't carry any particular baggage between the literal meaning.

Expand full comment

So, if you believe in sex not gender, what should an androgen insensitive person write? Given that they could likely appear female to anything but a genetic test, and live their life in that manner. As a corollary of that, have you ever had your chromosones tested? Do you know that you are in fact the chromosonal sex you are based on your phenotype?

Expand full comment

I mean, to take devil's advocate, first of all they could just believe in phenotypal sex, and in that model there would then not be a problem with having "other" or "intersex" as a category.

Expand full comment

This is what the "were you assigned at birth" language sorts out, doesn't it? An androgen insensitive genetic-XY person was almost certainly assigned female and had that on their birth certificate and so can say "was assigned female" - if they're happy with that, "cisgender woman" as in "gender identifies as woman, was assigned female at birth" is completely appropriate

Expand full comment

Had a good chuckle at this one because I realized I have no idea which direction you’re being triggered from

Expand full comment

"All the gender ideology stuff" being the single question about if you're trans or not? Unless it's been edited since you wrote this.

Expand full comment

Not him, but I personally dislike the "assigned at birth" phrasing. It connotes (in my opinion) an arbitrariness which isn't really accurate 90+% of the time. We say that (for instance) people are assigned to a specific customer support person. Or at work, we say that we were assigned a particular desk and workstation. In each case, it describes an executive decision made by management for how things will go in the future.

Picking which sex to put a birth certificate isn't like that. The doctor (or midwife or other responsible person) is making a judgement call about which typical sex binary the baby is closest to matching (which is occasionally a difficult choice, and sometimes made incorrectly.) But we don't typically use the word 'assigned' for a situation like this. We use the word 'categorize'.

I prefer the phrasing "how were you categorized at birth?"

(That said, 'assigned at birth' is a common usage and not generally worth fighting. I suspect the OP's actual issue is that it's an unambiguous tribal symbol for the blue tribe.)

Expand full comment

The "categorized" phrasing is probably more accurate, but the common understanding assumes that the categories are natural and objective (which for intersex people, they aren't). Assignment entails some arbitrariness, which is appropriate (not that gender assignment is totally arbitrary, just... a little. In some cases).

Expand full comment

I agree. Agreeing to be labelled as 'cisgendered' is acquiescing to a use of language that I don't want to. In the same way that someone (forget who) in response to the notion that one should be able to choose one's labelled gender (er 'woman' not 'trans-woman') said 'man' not 'cis-man'

Expand full comment

My opinion on this is that any holdouts will simply be co-opted into the agenda. You only want a category "woman" or "man"? Perfectly fine for the trans types who want everyone to agree that there is no difference at all - "a trans woman is a real woman", not "a trans woman is a trans woman", after all. A biological male who had surgery and is loading up on hormones will be the very same thing as a biological female under such categories, and the language-police you disdain will see that as a victory for them, not for you.

If I have to pick "cis" out of "are you cis, trans, other" then I'll do so because I do insist that yes, there is a difference between natally female and female via surgeries and hormones so for me there is a difference between "cis" and "trans" (and the "assigned at birth" language annoyed me as well, but again, I'll go with that rather than surrender the field).

Expand full comment

I acknowledge that (for example) 'man' and 'trans-man' as categories has it's problems, the migration to 'cis-man' & 'trans-man' meaning I am 'cis-man' is without my consent being sought or given

Expand full comment

I happen to think “don't categorize me without my consent” generalizes pretty poorly as a “true” rule—and you might be interested to know that that's one of the problems I run into when communicating in some other highly genderqueer-etc.-friendly (I almost want to say -centric even though they claim to theoretically not be) circles! That is, people there taking it on as a social reality defense: “how dare you put gender in things where it doesn't belong”, or “if you think of me as anything other than *just* my declared gender you're causing problems”, and especially “if you have reason to identify a variant/specific case of category X and then add *anything* to a term indicating ‘someone who isn't in that subcategory’ and use it on me, you're marginalizing me”. Observing the symmetrical position here is… interesting.

This is without saying that the pattern has no use or is harmful in all cases—only that it generalizes badly, and the badness is especially salient here, because a hard “you may only analyze me using ontologies I approve of” will escalate any attempt at comparative analysis between groups that don't share ontology preferences on the relevant information into “don't even try”.

Expand full comment

My impression of the trans community is that they do want to be treated differently from ordinary men or women.

For example, I am a cisgender man, who is also fat and has long hair (though not for much longer I fear). Back in my youth, strangers who only saw me from behind (thus unable to observe my beard) would address me as "ma'am" or some variant. Socially speaking, this is a minor faux pas at worst. However, if I were transgender, such an instance of misgendering would be treated as a severe insult, and potentially hate speech -- and my sense is that the modern transgender community would endorse this view.

Expand full comment

In my experience, they don't make a big deal out of honest mistakes. They'll just correct you.

Expand full comment

Should I start rejecting the label of 'democrat' because I think republicans shouldn't exist and don't want to acknowledge them as a legitimate political orientation?

Because I would love to start doing that, sounds like a lot of fun.

Expand full comment

More like rejecting the label of "democrat" because you don't have any political beliefs at all.

The people who object to "cisgender" don't believe "gender identity" exists (for them), so they object to the insinuation their biological body matches something that doesn't exist (for them).

Expand full comment

Thanks, your ideology relating to these topics will not be recorded.

Expand full comment

On the weight loss survey if I lost a lot of weight due to a untreated medical condition, (then gained that weight back when the condition was treated), I'm guessing it'd be more appropriate to answer "no" to "have you ever lost more than 20lbs under a year" question, even if the answer is technically "yes"?

... I'm guessing "have untreated Crohn's Disease and c. diff for 6-8 months" isn't the sort of hot weight loss tips the survey is interested in.

Expand full comment

I wasn't sure how to answer in relation to pregnancy either. I answered honestly, but I think it will seem inaccurate.

Expand full comment

I ran into the same issue. Pregnancy seemed not to have been anticipated by the responses available

Expand full comment

Same.

Expand full comment

Survey 21, optical illusions, would not open with the error "A problem occurred:

Couldn't find cookie for study result ID 31188. Are cookies allowed in your browser?"

Expand full comment

I experienced the same error on all of Chrome, Safari, and Firefox.

Expand full comment

Fine on Firefox on Android

Expand full comment

I got the same error in Firefox on both a Windows computer and an Iphone.

Expand full comment

Forgot to mention, I tried on Safari and Chrome on MacOS

Expand full comment
author

Weird, it works for me (though it's very long and annoying). I'll email the author.

Expand full comment

Worked, but way too long. I abandoned it

Expand full comment

Yeah, had to abandon it as well. Please remove from the list. Did a lot, but it was hard on my eyes, seemed repetative and there was no indication at all how many where left :-(

Expand full comment

I felt exactly the same way. Sorry, but I have kids wanting their supper, and I really need to know approximately how long this is going to take.

Expand full comment

Yes, they should at least indicate roughly how long it would last. I gave up as well and had annoying visual after-effects for a bit making me slightly woozy

Expand full comment

I got the same on Chrome.

Expand full comment

Maybe if anyone *can* see 'the test', *that* is the optical illusion!

Expand full comment

I now realize that the correct answer to the metaphysics survey was "what survey?"

Expand full comment

I'm getting the same problem on Chrome, including incognito Chrome.

Expand full comment

I'm one of the many for whom the same thing happened. Tried in Firefox on FreeBSD and Chrome on Windows. Adblocker enabled in both, which I guess might be relevant?

Expand full comment

Same issue in Firefox, with adblocker enabled.

Expand full comment

I tried on Chrome, Opera GX, Firefox, and *Microsoft Edge*, no luck :(

Expand full comment

You tried Edge? That's *real* desperation! 😁

And this comes from someone who clung on to using IE to the bitter end until finally they forced me off with "Edge now, no other options" and made me finally move to Chrome.

Expand full comment

On the other hand, I'm someone who NEVER used IE as my primary browser at any time; I stuck with Netscape 4 long after it was obsolete until Mozilla finally got stable enough to use as my main browser; then stuck with the Mozilla Suite/Seamonkey for a long time until finally switching to Firefox which I continue to use.

Expand full comment

Same, on my chrome copycat, aka "brave browser"

Expand full comment

Sorry, I think what happened here is that Scott posted the wrong link, since the one that I sent was a generic one that changes into a user specific one once you actually click one it. Now the study think that one very annoying person would try to do the study very often :)

With this link, it should work:

https://jatos.mindprobe.eu/publix/1521/start?batchId=1702&generalMultiple

Expand full comment

Holy moly is your survey long. I’d have appreciated some kind of progress meter telling me how far through it I was.

Expand full comment

Yes, it´s an experiment rather than a survey, those are annoyingly long most of the times (since we need a lot of trials to account for the high variability in responses). Sorry for that, I´ll try to insert a progress bar in between trials.

Thanks for perservering anyhow!

Expand full comment

I was about 30 minutes into the survey with no end in sight, when I quit. Since I didn't make it to the end, I couldn't give feedback/ask about the study, is there any way I can do that, that is not a comment section?

Expand full comment

Yeah me too, I didn't have that much free time, I probably did about 20 images and then quit.

Expand full comment

They're not kidding that it's very long! I think you should give a break about half-way through to let people rest their eyes; I found that constant staring made the images start to 'burn into' my retinas and interfered with the following images.

It was interesting in that there was definitely one image that would not resolve into uniformity for me, while a few others did very quickly.

Expand full comment

Same here -- there was always that one problematic image that would keep popping up. Also, some of the images did cause an optical illusion for me, where the red cross would appear to float in front of the monitor.

Expand full comment

Finished the experiment. I'll note that with the exception of the blue, I almost always (90%+) pressed the space bar 3-4 times before it accepted it, presumably because the peripheral image wasn't officially fully loaded yet.

Maybe I'm the only one, but it seems that if you're using the timing of pressing the space bar for anything, then gathering the time before the outside is completely faded in might be more useful information. As it is, for almost all of my responses, you're going to be measuring where I was in the timing of hitting the space bar and then releasing and hitting it again, rather than anything more meaningful, because the image became uniform before I was allowed to hit the space bar.

I presume you've taken it yourself to see how it works in practice?

Expand full comment

+1, except less than 90% -- maybe 30%?

Evaluation should give special treatment to space bar presses that were very close to the end of the loading time, they could be the n-th repetition already.

Expand full comment

I also had the experience where pressing space did not always immediately move to the next screen. I was not sure if it was intentional to the experiment, a bug in the experiment, or an issue with my keyboard.

As it didn't suggest I should press it more than once, I didn't, with the exception of a few where, after pressing it the first time, it looked like it stopped matching the peripheral, and then the peripheral moved back in sync with the image again.

I'll note that because this effect, there were some points that I hit 'Q' because of looking away from the red cross _after_ hitting space.

Expand full comment

Yup, that works. I agree that if it won't mess up your experiment it would be good to have some sort of indication of progress. (Maybe just some words every now and then saying things like "You're about 1/4 of the way through.")

It seemed like some of the images appeared non-uniform, at least for some time, for a reason you may not have intended: after-image effects from the initial image. (Of course, maybe you totally did intend that, but I wanted to point it out in case not.) I would be entirely unsurprised if there were cases where the inner and outer regions genuinely matched perfectly but I didn't press the spacebar because the inner region still looked darker/brighter/less-blue/... because stuff was shown there sooner.

Expand full comment

Oh, while I'm making suggestions: as a reward at the end it would be nice if you linked to a list of what the images actually were, or a thing that lets you do the experiments without contributing to the statistics -- having tried my best to be Good all the way through and therefore _not_ see properly what was really going on at the edges, I'd have liked to see what it was I'd been missing or not missing in each case. Even better if you gave a bit of information about what effects you were looking for (perhaps with a note saying "please don't tell other people who might do the experiment later").

Expand full comment

I really wished for this as well.

Expand full comment

+1

Expand full comment

The webpage became unresponsive after about 3 images - the screen was just a red cross on a white background with nothing happening for 20+ seconds, and neither spacebar/Q did anything.

Expand full comment

Yeah, I got that, except I stared at that one red cross for over ten minutes before finally coming to the comments section to see if that was what was supposed to happen. :(

Expand full comment

How do I enter my key at that link? I didn't have a dark room handy for this survey so I haven't tried it yet.

Expand full comment

21 seems to be broken - "A problem occurred:

Couldn't find cookie for study result ID 31188. Are cookies allowed in your browser?"

Expand full comment

Have the same problem

Expand full comment

Same.

Expand full comment

Sorry, there was some link mix up.

With this link, it should work:

https://jatos.mindprobe.eu/publix/1521/start?batchId=1702&generalMultiple

Expand full comment

I don't know what to answer in Survey 0, Demographics, for the psychological conditions where I'm both guessing I might have it (though not formally diagnosed) and I know I have a family member who's been diagnosed.

@Scott, I'll wait several hours before completing the survey in case you want to clarify which I should answer?

Expand full comment
author

Just check that you're guessing you might have it.

Sorry those aren't multi-check boxes, Google Forms is really bad at letting you analyze multi-check boxes and I try to avoid them whenever possible.

Expand full comment

Tip for future surveys: Specify that the survey-takers should choose the first option that applies to them if multiple options apply.

Expand full comment

Survey 5 lists "Bupropion" and "Wellbutrin" separately. Is there some reason to do this, e.g. does Wellbutrin differ from generic bupropion in its effects?

Expand full comment

I was curious about this, as well.

Expand full comment

The survey should list the brand name as well as the generic. People may know the former but not the latter. I Googled them, like a good little boy/girl, but I shouldn't have had to.

Expand full comment

I thought it might be a technique to figure out how savy/informed a person is, but they don't seem to do that with other drugs.

Expand full comment

For family members with mental illness, what is considered family? Sister? Uncle? Cousin? Or strictly two generations of ancestors?

Expand full comment

More specifically, I assumed that only blood relatives counted, but it doesn't say so.

Expand full comment

first cousins are within two generations (shared grandparents), second cousins aren't

Expand full comment

Given that this is the summer, survey-takers in between (HS/university/grad school) and (university/grad school/fulltime employment) are especially likely to be in weird in-between states with respect to their occupations and residence statuses. Any preference on whether to answer as one's past or future self?

Expand full comment

If you are going HS->Uni, HS->HS or Uni->Uni I'd say you're still a student. If you're going Uni-> job but haven't started work, you're unemployed (sorry!)

Expand full comment

I answered as my future self, even though it'll take a bit for me to actually get to studying my chosen field. Mostly because my past self didn't really have a specific area of study.

Expand full comment

I have an identifier from a previous survey, I assume I should reuse that? The instructions don't explicitly say to use your old identifier if you have one!

Expand full comment
founding

Alert: Survey 2 doesn't ask for user ID

Expand full comment

That doesn't seem to be the only one; #7 doesn't either.

Expand full comment
author

Yeah, I told the survey makers this but they didn't seem interested in fixing it, so I guess they won't have that demographic information.

Expand full comment

Survey #15 also does not ask for ID.

Expand full comment

Trying to do the weight loss one but I can't answer whether I have ever lost more than 20 lbs in a year as the side-effect of a medication or for some other reason because the answer is both. This seems to assume that if you have ever lost more than 20 lbs in a year, it only happened once. I suspect that is not the case for a lot of people and certainly isn't for me (for me, it's mostly because of sports, but at least once, it was actually due to medication side effects).

Expand full comment

As a point of reference, if you pay attention to NBA basketball or NFL football where skinnier players bulk up to get strong in the offseason, but struggle to keep that on during the season, it really isn't uncommon to see the same guy lose 20 pounds in a year every year for 15 years in a row. For weight class athletes, they might lose 20 pounds 10 times in the *same* year.

Expand full comment

The instructions say, "If you lost a lot of weight more than once, pick he most extreme case."

Expand full comment

Sure, but how do you treat that when it's roughly the same amount repeatedly? The in-season, off-season cycle, or fighting-weight, training-weight, tends to involve gaining and losing the same amount over and over. No single event is necessarily more extreme than others.

Expand full comment

Survey 13 includes mescaline in the list of "psychedelic tryptamines", but mescaline is a phenethylamine, not a tryptamine. This makes it unclear whether people with experience with other psychedelic phenethylamines (2C-B, DOM, etc.) should include them in the "other" category.

Expand full comment

Likewise for LSD.

Expand full comment

I'm willing to give LSD a pass as an honorary tryptamine. It does contain the tryptamine skeleton inside its ring structure, so it can be analyzed as a highly substituted tryptamine where the substitutions include additional rings. It is also covered by the Shulgins in TiHKAL.

Expand full comment

Yes, I had the same concern. I left out any non-tryptamine psychedelics (excluding LSD and analogues if we want to be that fussy)

Expand full comment

Survey 19 has the polarity of the answers switch halfway through the survey with no notice (ie, 1 switches from “strongly disagree” to “strongly agree”). I suspect the answers after that point are going to be spoiled unless that’s fixed.

Expand full comment

I noticed that! I nearly went the wrong way round and I wonder if that was deliberate to make sure people hadn't just picked a number and were filling in by rote?

Expand full comment

I wondered if that might be intentional based on the survey content.

Expand full comment

It's good survey design to mix up the polarity question to question (and user to user, if you can get away with it). Unfortunate that it might come after a series of questions that don't do so, but such is life - hopefully it'll wash out with the randomized survey order.

Expand full comment

Why is that good survey design? Doesn't it increase the risk of people answering the opposite of what they mean?

Expand full comment

Granting that you don't want to establish an expectation and then flip it halfway through like that one did, it's nonetheless important to shuffle around the presentation to fight against directional biases in response. It's similar to how a good election ballot randomizes the candidate order *per ballot*, because we know that being first on the ballot is automatically worth a few percentage points.

Expand full comment

Ditto

Expand full comment

What's the deal with user ID? Where can I get one? Is it necessary?

Expand full comment

There's a link to a random ID generator in the first survey.

Expand full comment

What's to stop you from using a nonrandomized User ID? Why is a randomized one required?

Expand full comment

Nothing stops you and it isn't required. But a random one is better at maintaining anonymity and less likely to be picked by multiple people.

Expand full comment

For me, survey 21 says: A problem occurred:

Couldn't find cookie for study result ID 31188. Are cookies allowed in your browser?

I have cookies enabled.

Expand full comment
founding

Some commented elsewhere in this thread with a corrected link.

Expand full comment

I'm a bit unclear on what I should put down as my work status since I'm a student but am also working at a convenience store. I guess I'll put student?

Expand full comment

The digital literacy survey has some typos and grammatical errors.

Expand full comment

Completed all of the surveys that apply to me, other than the optical illusion survey which doesn't work. I consider this sufficient to boast/worry about, as per the post's suggestions

Expand full comment
author

Congratulations! (also, you're *fast*)

Expand full comment

I had to leave out the fun ones about psychedelics and being trans and so on, because I am Very, Very Boring And Dull.

Still, I hope I can boast/worry in company with my peers here!

Expand full comment

Same! We should get a medal.

Expand full comment

Wow! If you want to complete the last one also, try the fixed ink :)

https://jatos.mindprobe.eu/publix/1521/start?batchId=1702&generalMultiple

Expand full comment

Same (I did get the optical illusion one to work for me and I feel doubly proud to have sat all the way through that one.)

Expand full comment

Same here. When you do it over the course of few days, it's not much at all.

Expand full comment

I just took all of em (except the ones that don't apply to me) in a blatant bout of procrastination from packing!

Expand full comment

I find the political spectrum question tricky because I consider myself a centrist in my country but that would put me very much on the left in the US.

Expand full comment

The people asking questions about US politics will probably just exclude non-Americans anyway.

Expand full comment

I wonder - one thing that comes up in my conversations with Europeans is that while I'm a centrist in the US I would probably also be a centrist in, say Norway (which would put me much further objectively left) because their government is better run.

Expand full comment

Survey 1's probability section asks for answers from 0-100 in percent, but requires a numerical answer with no other characters (and also no bounds on possible answers). This creates a lot of practical ambiguity if some people revert to answering on a 0-1 scale, since having all one's answers be less than 1% is not unreasonable.

Expand full comment

That one is poor, but since it was phrased in "things happening TODAY" terms and since I don't think cryonics (as we have it in the past, now, and the near future) is anyway workable, I gave all those a resounding 0.

*Maybe* in 50 - ? years time, something that is feasible will come along, but honestly? Right now? I think it's the equivalent of sticking Auntie Gladys into the chest freezer after she's popped her clogs and hoping some day something can be done about it.

Expand full comment

Yup. I just answered those with a 0.000001.

There should be a 'technically possible, but not happening' button on those

Expand full comment

50 years is way too early. I am not convinced cryonics is even theoretically possible, let alone technologically achievable.

Expand full comment

I entered decibel answers, and they weren't rejected, so I assumed it was alright? But I did find the wording confusing, and wondered if I should round to 0 on the smaller numbers.

Expand full comment

I object to the omission of "Anarchist" as a political orientation. Left-anarchism is very much not the same as Libertarianism, and it's even more distant from any of the other options.

Expand full comment

Seconded. There was no acceptable answer for me on that list.

Expand full comment

The metaphysics survey put Bakunin in with Marxists. I think you are right though. Very significant differences and anarchist should be on the list.

Expand full comment

I thought about that, but Bakunin was a socialist anarchist, who wanted things to be run by communes without a centralized government. Which means he had the same ultimate goal as Marx. I don't find Bakunin's position coherent--there's no way to create anarchy and have it come out all communes--which I guess influenced me not to give him his own category. I would similarly have grouped an anarcho-capitalist with libertarians. They're different from libertarians, but they're more different from other types of anarchists. It seemed to me that I'd have to add at least 3 different categories of anarchy to the list in order to have any hope of picking out the value "anarchy" via that list.

I should've probably added a separate question about anarchy and self-organization. I wrote one about whether humans are generally good or bad, but it got so bogged down in different options and meanings of "good" and "bad" that I gave up and left it out.

Expand full comment

I think this is something you should keep in mind when doing your analysis of the results. You were fairly opinionated in which alternatives you allowed, and how you interpreted the positions you are asking about. In this case; maybe it is not so interesting that _you_ don't find X position coherent, if there are those among your data subjects that do. Perhaps it would have been better to stick to the SEP/googleable definitions and position enumerations. Especially since there often were no "none of the above" alternatives, I think you will have some noise and missing answers to deal with.

Expand full comment

I was hoping people who wanted to select "none of the above" would skip the question. It would make data analysis technically simpler. But it turns out very few people skip questions.

Expand full comment

#21 does't load; the message is "A problem occurred:

Couldn't find cookie for study result ID 31188. Are cookies allowed in your browser?". I do allow cookies. I'm on MacOS using Safari, but at a guess this is the result of a bad URL rather than a browser issue.

Expand full comment

Same result here, on Chrome on Linux.

Expand full comment

Same here, Chrome on Windows 10, even after I turned off ad-blocking software.

Expand full comment
founding

Someone else commented in this thread with a corrected link.

Expand full comment

For the mental health conditions in survey 0, I assume that if I've had a diagnosis for a condition before but wouldn't qualify for it anymore, that still counts as "I have a formal diagnosis for this condition"?

Expand full comment
author

Yes.

Expand full comment

I had this question too. The survey doesn't distinguish between a person who is currently depressed or suffering from OCD vs. a person who had that problem decades ago.

Expand full comment

If this survey opportunity ever comes around again someone should make one about how psychiatric diagnoses evolve. If you had diagnosis x at year 0, by year 1, what was the primary diagnosis, and year 5, and year 10.

Expand full comment

Oh, whoops.

Expand full comment

On 6: weight gain/loss, there's no question about having given birth some time before especially a gain, which is often relevant

Expand full comment

Surely an even more relevant question is giving birth immediately after it? Certainly that was the mechanics of my own weight gain...

Expand full comment

I have a question about the demographic question about depression. If I both believe I have un-diagnosed depression and have a family history of diagnosed depression, which answer do I give?

The question is:

"Depression

Regarding clinical depression...

(a) I have a formal diagnosis of this condition

(b) I think I might have this condition, although I have never been formally diagnosed

(c) I have family members (within two generations) with this condition

(d) I don't have this condition and neither does anyone in my family"

b and c both apply. Some other demographic questions have this issue as well.

Expand full comment
author

I would default to the "highest-level" answer, so B.

Expand full comment

The questions on the Metaphysics survey are so hard to respond to that I gave up. For example, the question: "How certain are you that racism is incorrect?" Racism is an act, not a proposition! How am I supposed to answer that in a meaningful way? Many of the other questions were ambiguous or didn't have an option for the answer I would actually give.

Expand full comment

That one clearly means “how certain are you that the beliefs and supposed ideas that are generally considered racist / associated with racism is incorrect”

Christianity, or democracy, or homeopathic medicine, or science, can all to various degrees be behaviors, yet they all have clear “fact” or “agree disagree” components to them just as racism does. Also racism refers to a spectrum of beliefs ranging from white supremacy to races having different genetic iq to “blacks are more violent” to “a society with whites having on average more power slash positions than blacks is fine and not extremely evil” to “some parts of modern black culture maybe sucks a bit” rather directly - Richard Spencer is a racist - racism can refer to the state of being racist - not because he’s done any “acts”, but because he believes blacks are dumber than and should be separate from whites or deported - and that’s not an action, it’s a belief.

This is probably a dumb thing for me to argue but I just don’t get why one can’t go from “racism is an action” to “well even if that’s true it’s an action that’s closely tied to a set of beliefs and depends on said beliefs, and the question is asking me about that “

Expand full comment

So you would answer “yes I am certain” if you’re certain what is seen as racist beliefs by whoever are false, and no if you think many racist beliefs are true, and maybe if you don’t know about some or think some beliefs seen by extremists as racist maybe aren’t, but most are

Charles Murray or Steve Sailer would not be certain at all and quite certain racist beliefs are incorrect, an IDW figure might be eh depending, and a normal progressive would be certain.

It’s like most survey questions very ambiguous but not really that bad. Maybe should’ve referred to what sort of racism, but maybe that’s part of the intended question

Expand full comment

Taking a look and that’s easily one of the least confusing questions

Expand full comment

I have to disagree. I think in most people's minds, 'racist' has specifically moral connotations; it has a sort of meta-definition of 'unfair treatment on the basis of race.' But they disagree over what treatments meet that definition (and thus there is no set of ideas 'generally considered racist'; many people consider affirmative action racist; many consider the absence of affirmative action racist. I think both groups agree on what affirmative action is, but the reason they disagree with whether it's racist is because they disagree with whether it's moral). So for many people, asking if racism is incorrect is a bit like asking if evil is incorrect. If I think something evil is correct, then must not think it's evil.

Expand full comment

Those people answer “certain it’s immoral / certain it’s incorrect” then

Expand full comment

HBD leftists, or Charles Murray, might answer “certain it’s immoral / not at all certain it’s incorrect”

Expand full comment
author

"The questions on the Metaphysics survey are so hard to respond to that I gave up."

That's how you know it's some high-quality metaphysics! Though the one I always hear is "Describe Existence, and give three examples".

Expand full comment

I’m just enjoying pondering them

Expand full comment

"Though the one I always hear is "Describe Existence, and give three examples".

Oooh, never heard that one, but I'll give it a bash:

(1) God the Father, First Person of the Trinity

(2) God the Son, Second Person of the Trinity

(3) God the Holy Spirit, Third Person of the Trinity

All other existence is contingent 😀

Expand full comment

I protest; those three examples are just one example!

Expand full comment

That's modalism, Patrick!

Expand full comment

Hahaha. Excellent answer!

Expand full comment

Not really, you can examine complex and confusing questions, but describe them in a more or less clear way. It should be hard because of what its asking, not because it is hard to reverse engineer what the author was trying to say

Expand full comment

They’re not clear, but the way they’re not clear is at least interesting. Good luck describing the art one clearly, or the one about “can a society’s beliefs be wrong about its own terms”

Expand full comment

Bullshit. Half those questions depended on the definitions of the words used to ask them. I have very little tolerance for people who communicate themselves poorly then act smug when misunderstood, and that’s the vibe I got throughout the entire survey.

Also, it’s leaking the results of the survey! Literally anyone can download all the previous responses! You need to take it down until its author fixes this.

Expand full comment

I agree, most of them are more or less meaningless with that little clarification - however most people manage to have strong opinions regardless, and the topics themselves are interesting if sometimes only In the sense of “why do other people think this means anything”! That said, metaphysics is hard, and who would answer a survey with each questions prefaced by five thousand words worth of Stanford encyclopedia of philosophy articles? If you’re interested in clear and precise (or at least long and autistic) definitions of metsphysics concepts, SEP is a fun read by the way.

Expand full comment

I didn't want to get people's opinions on questions phrased using my own long, carefully-specified, unwieldy definitions; I wanted to learn about how they interpret and interact with public discourse. People in public discourse /don't/ clearly define their terms; they say "racism", "progress", "justice", and so on, and expect people to understand them. So it seems to me that knowing what someone thinks about "racism" has more predictive value than knowing what they think about a precisely-specified concept which never comes up in public debate.

And in any case, I couldn't decompose definitions down to some ultimate objective level, because the objective level would be a large collection of sense data. Since my definitions would have to bottom out somewhere still in the airy subjective realms of abstraction, it seemed pointless to spend a lot of effort defining subjective terms in terms of other subjective terms.

Expand full comment

"When one door closes, and another one opens, it means you have poltergeists." - Taika Waititi's twitter

Expand full comment

Open Thread: Metaphysics Edition - where the comments are dedicated to discussing particularly interesting “pop metaphysics” questions

Expand full comment

"Is racism incorrect" is definitely the hardest one to interpret, but the others I could intuit reasonably well. Maybe it means "How certain are you that racist beliefs are incorrect?"

Expand full comment

I think the difficulty lies mostly in that what is considered racism varies a lot. Something that's considered racism in San Fransisco might just be considered common sense in Plovdiv, or whatever.

Expand full comment

Yes, this. If "racism" is the kind of thing found in Kendi's work, then probably most of it is correct/fine. If it's the kind of thing even my parents would call racist, then it's going to be a bit extreme and bad/wrong. There's a pretty big spectrum left to interpretation.

But I suppose the questioner might be trying to infer the number of people who think of "racism" as defined in CRT terms while still disapproving of that conception. That seems like the main way to arrive at a "racism is correct" answer.

Expand full comment

Huh. I found the more abstract and metaphysical ones much tougher than the racism one, which at least clearly corresponds to various ideas that can be cleanly evaluated, even if it’s not clear which. Whereas something like “science proves or disproves statements about the physical world with certainty” - much more nuanced and strange. At least the first one I’m pretty sure I can guess how most normal people would answer it, whereas the second I’m guessing most people will answer it one or another way for quite different claimed reasons.

Expand full comment

It's nuanced and strange, but as a scientist, I'm pretty sure it's false. The rock/Rube Goldberg/car/butterfly question doesn't have a clear answer but whatever you answer, you can easily justify it. The tv watching question clearly does not encompass all options. The snail question I just kind of intuited.

The two racism questions, however, are so close to each other that I can't intuit the answer or justify it on both at the same time.

Expand full comment

Also, how most people would answer it doesn't seem relevant to your own answer. That they will have different claimed reasons is probably the point of including them in the questionnaire.

Expand full comment

Well that just means it has a determinate answer in many cases - Andrew Anglin would answer one way, Bluehair SJW answer it another way, and we can understand why. Whereas when you say that’s false, you’re probably saying that for very different reasons than others and who knows why!

Expand full comment

Do you think that “light is an electromagnetic wave that is emitted by hot objects” has not been proved certainly?

Expand full comment

Strictly speaking, science can only make judgments about the likelihood of whether a statement is true or false. The likelihood that your statement is true can approach 100% but never reach it by any way short of hacking into the source code of the universe. By that criterion, it has not been proved certainly.

It might seem like an absurdly high standard but on anything more complex than classical physics, it's important to remember that even the most solid scientific results are merely likely, and not guaranteed, to be true.

Expand full comment

This is one place where I'm not a Bayesian. Science says "this is the best explanation we have now" or "here are the explanations that some reasonable scientists consider plausible currently", hopefully followed by "and here's what they predict". It doesn't give odds of correctness.

Expand full comment

Yeah, that one was a bit tricky too. Intuitively I thought "false", but then I quickly realized that interpreting "certainty" as an absolute was a really bad idea and changed my answer.

Expand full comment

I went the other way. If „certainly“ isn’t absolute then why include the word? So no, science doesn‘t prove anything with certainty. I‘m still 100% pro science. So I guess the results won’t be very helpful.

Expand full comment

I found it easy; science DOES prove or disprove statements about the physical world with certainty, at any given moment.

Newton was right, then Relativity was right, and when we get some TOE, it will be right, and then the next thing will be right also.

Expand full comment

Didn’t science prove the „phlogiston causes fire“ thing? I‘d say: Science itself allows to make statements (predictions) about the physical world. So the better the prediction is, the better my certainty becomes. But it will never be „full certainty“.

Expand full comment

I found it a bit tricky because I would agree that any observed negative would definitively disprove an absolute positive but not the other way around, i.e. science can in my view disprove things (to my satisfaction) but not prove them

Expand full comment

Yeah, I just answered as though they meant 'IBD'.

Expand full comment

I think actions can be correct/incorrect ways of accomplishing goals.

My first impression was to take it to mean something like..."if the reasoning that leads to believe that insert-race-here are inferior in insert-some-dimension-here is true, is acting upon this belief by deporting/subjugating/de-prioritizing/racist-ing the incorrect action to take?"

Expand full comment

But the previous question was "Is racism bad?", and the answer to that would be what you're describing.

Expand full comment

Hmm, I don't think that is correct. My description has little to do with good/bad. It has more to do with goal-fulfillment. You could say whatever racist acts are bad, but still think they are correct to fulfil the goals of whatever your model of the spherical-cow racist is.

Expand full comment

> Racism is an act, not a proposition!

What do you mean by that? I've always heard that racism is the belief that a) humanity is composed of different races and b) you can order those races on a mostly one dimensional scale.

Expand full comment

That's the most bizarre definition of racism I've ever heard. What does "mostly one dimensional" even mean?

Expand full comment

"mostly one dimensional" here means race X is better than race Y which is better than race A and B. I'm curious, what is your definition of racism, and what makes mine bizarre?

Expand full comment

Because you leave out what the dimension is, and what it means that something is *mostly* one dimensional. Can it be 1.5 dimensional? Is it 1 dimensional 90% of the time and 2 dimensional 10% of the time?

Also, how is "better" a one-dimensional scale? Even the most principled racist will admit that Ethiopians are better at Olympic sprinting than Scotsmen, but that's surely not the one dimension you mean.

I would agree on your a) but instead of b) I'd say that racism is the belief that a person's race determines their character* to a significant degree, such that it's justified to make generalizations of a group of people based on their race.

*Or personality, temperament, mentality, whatever term you want to use.

Expand full comment

> Even the most principled racist will admit that Ethiopians are better at Olympic sprinting than Scotsmen, but that's surely not the one dimension you mean.

I don't think I agree here, for example, if you look at Nazi ideology, they believed that Aryans were superior people. There is no "X race can be better than Y race in Z case", but strictly "X race is superior to Y race". The dimension here is which race is better. I said "mostly one dimensional" because you'll hear exceptions sometime ("honorary whites" in the alt-right circles for example).

To understand this a bit better, would you say that your statement that Ethiopians are better at Olympic sprinting than Scotsmen is racism? Or wouldn't it count because either a) Ethiopian and Scotsmen are not races or b) being better at Olympic sprinting is a physical reality that doesn't count, or even something else?

Expand full comment

My example was to illustrate that "better" is way too imprecise a term to use in your definition of racism, but your further explanation illustrates that you're not thinking broadly enough. We're not talking about Nazi ideology: Nazis may be racists but not all racists are Nazis. Racism also isn't the same thing as white supremacy. You seem to be limiting your definition as racism inflicted on colored people by white people, and ignoring racism inflicted on Bangladeshis by Indians, on Koreans by Chinese, on Peruvians by Chileans, etc.

In short, racist beliefs do not equate racist ideology. Most people have racist beliefs about certain groups of people (almost always their direct neighbours) but they don't keep a ranking of how their race compares to all the other ones in the world, or build their worldview around such an idea.

Expand full comment

Not that it changes the substance of what you're trying to say, but Ethiopians are good at distance running, not sprinting.

Expand full comment

Suppose that a Blackfoot Native American respects the Cree, sees them as just as good as the Blackfeet, and believes it's his duty to kill every Cree he meets and help wipe them off the face of the earth. This is my impression of what some Native American inter-tribal relations were like in the 17th and 18th centuries. Is that Blackfoot racist?

Expand full comment

Do they act on their belief to kill every Cree they meet? Why do they believe that they have to kill them? But from what you've said, I wouldn't call it racist.

Expand full comment

The example isn't realistic, and certainly not representative of all Blackfeet. The Cree and Blackfeet were traditional enemies in the 19th century, but they could meet each other without violence. Nowadays I expect they get alone well, since they both have the same conflicts with the government of Quebec.

Expand full comment

I think taking a non-realist example isn't helping here. The goal with defining racism is to analyze real-world attitudes and do something about them, not to imagine all kind of weird combinations of morality people could have that will bend your definition. It's a human definition so it's not perfect by default. It's not maths.

> The Cree and Blackfeet were traditional enemies in the 19th century, but they could meet each other without violence.

So it's like many countries in Europe before our times. Again, I wouldn't call it racist.

Expand full comment

The question I'm trying to get at is whether "racist" means something closer to "believing your own race is superior", or "considering your own race more important in your moral calculus".

Expand full comment

There is some really interesting scholarship on Comanche kinship patterns. Raiding and kidnapping was a thing in multiple indigenous societies in what is now the US west. But the kidnapped people were often adopted and became part of the society that kidnapped them. The "kill them all" strategy did not seem to come up as much. The "you will be part of my group now and do things my way" strategy did come up.

Expand full comment

Is 'adopted' a euphemism for 'enslaved' here?

Expand full comment

My impression is that it is not, with the proviso that raiders were more likely to kidnap females than males, and so you might want to look at the status of women in the raiding tribe to answer that question more fully.

Expand full comment

I smiled at that one because I think it was intentionally worded that way. Made me stop to think, at least.

Expand full comment

I found it particularly frustrating that it didn't distinguish between normative beliefs and descriptive beliefs, which makes a lot of the questions meaningless

Expand full comment

I had trouble with that question for a different reason. I actually have no clue what exactly the word "racism" means. It's a totally nebulous concept.

Expand full comment

I don't know if 'nebulous' and 'has a thousand different highly precise definitions, each used by different people in different contexts and different times in history' are synonymous or antonymous.

Expand full comment

Having a bullet-proof definition of "racism" would be a good start. Here in Australia, there are many indigenous activist groups working tirelessly to implement Aboriginal apartheid and segregationist programs (race-based parliamentary representation, segregation onto "Aboriginal only, Aborigine-controlled" bantustans, Aborigine-only government funding and programs, preferential hiring status, separate hospital admission protocols, etc etc) that, to me and many others, are self-evidently racist, but which have widespread support amongst the "(so-called) "anti-racist" fraternity. I also see fascist paramilitary groups like BLM and Antifa as inherently just as racist as the (so-called) much smaller "far-right" militias, as is CRT as a concept, very popular at the moment amongst the elites. I have my opinion on what constitutes racism, they have theirs, and it's dipolar.

Also, disentangling "like/dislike of particular cultures" and "like/dislike of particular religious belief systems" from "racism" would be good progress.

Expand full comment

Yes, a lot of the questions on this one were ambiguously worded. I think the standard request is 'make up what you think the question is trying to get at and then answer that question instead,' which is annoying, but sometimes necessary when asking about complex topics that people will have different conceptions of.

Expand full comment

I think I actually did complete all of the ones that applied to me. What can I say, slow day at work.

Expand full comment

I wish #4 had at least one free form text input field, and I suspect my answers will appear inconsistent to its author.

To summarize what I would have said: I know a lot about how most digital technology works internally, but experience the user interfaces as having been designed to be hard to use, hard to learn, and most likely to intentionally exclude older users, as well as those with a number of disabilities.

I no longer consider myself "digitally literate", since I can barely perform basic functions with the latest cell phone style of interface (no text, lots of icons, UI elements invisible until you make the right gesture at some unmarked spot on the screen, no help, no user's guide)

Expand full comment

I'm in much the same boat; I know the theory of the terms, but the notion of "do everything via smartphone" instead of desktop PC is beyond me (my bank is forcing everyone to move to exactly this and I *hate* it).

Expand full comment

> most likely to intentionally exclude older users

This surprises me. What would the motivation be?

Expand full comment

Sarcasm, in my case.

Leaving aside the intentionality, what I see is young designers who either don't talk to older people, or presume anyone as old as a boomer is too stupid and senile to understand tech, it not having existed in their youth. (Many young people seem to be weak on both chronology and history. But they also often consider technology (by which they mean "digital technology") to have started at what I'd consider some intermediate milestone, e.g. perhaps with the iPod ;-))

But realistically, I've no clue why they produce these tools that are massively difficult for those who started with much earlier tech to either figure out or like. I just observed that some of them say things like "everyone already ..." and turn out to mean "everyone I knew at the time, of whom all but my parents were under 30" ;-(

Expand full comment

Ahh, ok. I think it's like you say. They (ok we) (ok maybe not, I'm 43) can't (because they haven't tried) put themselves in the shoes of those significantly older.

Expand full comment

What’s particularly difficult in modern tech?

Expand full comment

A good example is the state of settings on Windows 10 compared to XP.

Expand full comment

>> no text, lots of icons, UI elements invisible until you make the right gesture at some unmarked spot on the screen, no help, no user's guide

Expand full comment

For one thing, they keep changing it. I've been using Windows for over 20 years, but it's not like it used to be, so I keep having to learn new ways of doing the same things.

For another thing, the changes tend to make things more complicated. A couple of years ago my microphone wouldn't work, and I had to take it to the shop to get it fixed. Turned out it was muted in the settings, but not in the microphone settings where I had been looking. This would not have happened 20 years ago.

Expand full comment

I don't use 'modern tech' much if that's defined as fancy telephones and tablets, but recently had some accelerometer data saved in an excel file on an iphone that I needed to retrieve. I plugged the phone into the usb port on my laptop and opened up the data folder like any other usb device. The only data I could access was the photographs. To get the excel file, I had to gain access to a cellular or wireless network and email the file. Why not allow me to easily access every file on the telephone through usb? I found this particularly difficult and unnecessary.

Expand full comment

Pre-personal computer technology had (I think) the assumption that certain aspects of the design had been converged on through trial and error over a long period of time. So small aspects of design variations carried meaning, because it was well-traveled territory. Things like a title page, or a dust jacket, or aspects of typography. With newer tech, the designs have not been converged on, the trial and error is very much still going on and so the tech is "read" differently by the user. Titles of nested menus do not have multiple levels of meaning. Things are different in unexpected ways. I worked in a school setting where they bought a lot of machines running chrome, that had no "hard drive." I don't want to tell you how many people couldn't turn them off and were embarrassed to ask how.

People who reached adulthood before PCs were not generally encouraged to noodle around with technical objects (unless with farm equipment, cars, a repair career or engineering). Ruining it somehow was a concern. So people carry residual shyness about poking around the machine looking for the off button. It probably sounds incomprehensible to the younger generations.

Knowledge always has patterns.

Expand full comment

> So people carry residual shyness about poking around the machine looking for the off button.

This is very very true for many older people who have difficult with technology. Couple that with all the stories on the nightly news about The Hackers, and many older people I know are deathly afraid of doing anything with tech.

Expand full comment

I've developed software for decades, and I don't think I'm scared of computers.

But gmail archives messages when I'm trying to delete them, and I've never found an undo function, or a way to access the arhice from a cell phone. On the other hand, Apple's iPhone email deleted random messages when I put the phone in my pocket (having turned the screen off) and the undelete function didn't work as advertised, possibly because you have to use it right away, without enough time to consult a friend or google the information.

Obviously the thing to do is for everyone to accept e-statements etc., especially if they come by this ultra-reliable (sic) email.

My theory is that Microsoft taught everyone that if any software works 90% of the time, it's of excellent quality, and users should rejoice. Only those of us who grew up with typewriters and adding machines - or alternatively, with mainframes and unix - expect better than that.

Expand full comment

'Cause they sell cell pones on the sea shore to Hype Beasts who will lay out 2k for the latest 5% more instructions per second per core piece of glass and aluminum as a status symbol, and design as such.

People who buy cheaper and less often are along for the ride.

(Personally, I fuckin HATE web interfaces when they get too cute, and if windows ever gets rid of the old control panel and details windows I will shit, but don't mind gestural stuff on cellphones.)

Expand full comment

> But realistically, I've no clue why they produce these tools that are massively difficult for those who started with much earlier tech to either figure out or like

Marketing. Modern UIs are designed by committee / by focus group. They are a collection of marketing trends that are projected to increase ROI, and not tools that are designed to allow the user to complete his work most efficiently.

Expand full comment

I've also more than once encountered designs that were the result of A/B testing, combined perhaps with measuring the wrong desired goal state.

As an example, if a web site becomes harder and slower to use, engagement first goes up, as each person trying to accomplish a task there takes longer to do it. It then goes down, as some portion of the previous users find better choices, or decide that the activity isn't worth the time it now takes. Naturally, it's the initial change in engagement that gets measured before the decision to go live with the new interface.

Also related, we have DuoLingo, where every change reduces the amount of useful information per screen-sized section of a page, requiring more scrolling, and/or requires clicking through meaningful content-free pages on the way. I presume there are ads in the extra space, except for those of us using AdBlock (me), and also that even those who pay for an absence of ads don't get to use the older, faster UI. They've done a great job of making me into a very occassional user, since no amount of gamified achievements can compensate for the annoyingly slow site.

Expand full comment

Tangentially related: one of my friends used to work at google, and he said the designers (all of them on the younger side) didn't understand that light gray text on a white background is hard for older people to read and that it matters.

Expand full comment

It's not just software designers.

My credit union gave me a credit card stylishly designed such that I had major difficulty reading my credit card number - probably because I've had cataract surgery, and their design interacted badly with one of the subtle changes resulting from it.

The numbers were slightly raised, but not coloured at all differently from their background. Low contrast FTL.

To their credit, the replacement had the numbers very dark, as well as raised, and they were easy for me to see.

Expand full comment

I have a special hatred for devices where important icons are bas-relief in black plastic.

Expand full comment

I've heard it said that low-contrast credit card numbers are a security feature, designed to prevent strangers from snapping a photo of your card.

Expand full comment

Security features often interfere with legitimate use ;-(

Expand full comment

I can navigate annoying UIs, but I made the deliberate decision to opt out of most social media (except for this comment section and DSL), so I suppose I am technically illiterate as well...

Expand full comment

Hmm, I don't think DSL counts as "social media" - that term got redefined years ago to include nothing older than FaceBook and Twitter. I was on LiveJournal at the time, and was shocked to discover, when answering surveys, that I wasn't able to claim to use any social media, since I wasn't on any of the listed options.

But you have a point about being technically illiterate. I also don't do well with websites and apps that borrow UI elements from e.g. faceBook, expecting everyone to recognize them without any explanation.

Expand full comment

I did all the ones that applied to me except for two, where the formatting was counterintuitive for me and I kept making mistakes, so I didn’t want to submit with errors

Expand full comment

I almost did. Stopped at #19.

Expand full comment

That was extremely fun, but I have some questions.

On the Metaphysics survey, it went from page 1 of 3 to page 3 of 3. I checked, going back and forth, and it came up the same. Is there (a) a missing page 2 (b) ha ha, deliberate error, this is a metaphysical question?

On the Weight/Gain loss survey, it gave a great selection of "is this why you lost/gained weight" but nothing as simple as "went on a diet" or even "Other - lost weight because of this thing, gained weight because of that thing" which I do think is a flaw. I lost (and gained back) weight over a two-year period, but it wasn't because of "broke up with a boyfriend" etc. reasons

On the Covid-19 Impact, I skipped the wages question because (a) I can't remember exactly what I earned back in 2019 (b) I certainly can't be bothered trying to work out "if I worked X hours per week at Y rate per hour, what is that over a year?" and (c) none of your business, toots: if you want to know did I lose money during the pandemic, just ask that!

Expand full comment

(I wrote that survey, hi!)

Totally fair if you don't want to give answers on that one. Yes, I want to know if people's income went down over COVID, but I also want to quantify it (also perception and reality are not the same). If ultimately I don't get enough answers on those questions, I'll have to void some of my hypothesis, but that's fine.

Either way, thank you both for your feedback and for answering the survey!

Expand full comment

There was nowhere to mention voluntary work. Part of that is gone forever, and I am sad. The other part is worthwhile, and still going, although there were lockdown difficulties. Not all work is waged.

Expand full comment

Question: Does everyone have a race? It's a social construct after all. I'm sure the american society could possibly assign one for me, but I've never been there. Does it only go by apperance? Then why would we need another word for it? It also seems different from the concept of 'people whose ancestors came from the same place look somewhat similar'.

I know about the falsified scientific theories about different human races, but you probably don't want such bogus categories in this neat little dataset.

I don't really want to describe where my family and I come from and live, because then people might try to invent a race for me instead of answering the question ;)

Expand full comment
Comment deleted
Expand full comment

I'm not sure it has any coherent meaning in the US. You could give photos of a bunch of people of different ethnicities to 100 Americans and ask them to categorize them by race and I bet you would get WILDLY different answers all over the map. Same if you even gave them a precise break-down of their ancestry and DNA. It will always be true for any person that the further away someone is from the categories that are closely relevant, the easier it is to throw them all in a vast indistinguished lump -- say "Asian" or "white" -- while the closer and more familiar one is, the more readily one makes fine distinctions.

Expand full comment

Speaking as an American, yes everyone has a race, though people might disagree on how many races there are, or where to draw the line between them. If you came to America, people would try to figure out what race you are; "no race" would never cross our minds.

Expand full comment

I think it's morally abhorrent to determine rights or privileges based on one's genetics/ancestry, and I think most people agree with that.

I also agree that identification into a broad "race" is very reductive on a personal level, and very complex on a scientific level. Yeah, I think you can slice the human race into some large chunks based on their genetics, and there are some observable differences in those groups, though the differences occur on large spectrums that overlap with other groups. I think it can be useful to understand some of those general differences for a lot of practical reasons. For example, Black Americans are more susceptible to sickle-cell anemia and respond differently to blood pressure medication than Americans with European ancestors. Black Americans are less prone to depression than white Americans. Why? I want to know if it's a diagnosis problem, or if there may be some difference of genetics or upbringing that influences this.

I think getting too carried away with "race" leads to trouble, whichever side of, say, the BLM cause you might fall on. But I also think it's not helpful to pretend that people with origins in different parts of the world might (in general, and on a broad spectrum) have some differences. Calling those differences "good" or "bad" is where a lot of people have gone wrong, and I think the "race is a construct" idea is an attempt to unite us as one common human race (which is good and fair.) I wish we could get more "post-racial" in how we treat each other but we could still acknowledge and enjoy our diversity.

(Note - it's also interesting to see the dramatic difference between various African groups. I'd say there's more genetic difference between a Pygmy and, say, a Kenyan than there is between a Swede and an Italian, but they'd both be "Black" in America. Worth thinking about.)

Expand full comment

Although I generally answer "prefer not to say," it does seem somewhat interesting and relevant for this survey, since one of the big culture war fronts, which impacts topics of interest here, centers around race. Especially American white/black racial politics. It would be interesting to know if there are black ACX readers. If everyone is European descent/white, that might be interesting to know, though there are people who might automatically find that problematic.

Some racial categories are kind of odd, like someone with one minority great grandparent identifying as black or Native American or whatever, but for sociological reasons it's kind of interesting to know, nonetheless.

Expand full comment

Race is just the geographic origin of your ancestors. It's likely to be at least somewhat mixed for places that were colonized like South Asia, Africa, and the Americas, possible to be somewhat closer to the pre-globalization norms in places like Japan that have maintained more or less continuous isolation and rule of the same ethnic groups for at least the last five centuries.

You can check out the human phenotype project (http://humanphenotypes.net/) for some postulation based on historical records gathered by travelers and anthropologists of what this situation looked like before people learned to widely travel between continents and populations still maintained distinct regional variations. This project creates 38 basic types of human and more than 200 local variations. The US Census Bureau in the 1970s decided to categorize everyone into 4 races. How detailed you make the clusters here can vary, and people tend not to perform genetic tests in practice, and where they do, there are flaws in how well those tests can really determine your ancestry because they depend heavily on who else took the same tests, so yes, it is at least partially if not mostly based on appearance in practice because there may not be much else to base it on.

As the phenotype project points out if you read through, genotypes and phenotypes don't map neatly to each other because the regional qualities of the environment that cause particular phenotypes to arise happen in many regions that are isolated from each other, so plenty of people end up looking roughly the same even though they have no more genetic relationship to each other than to any randomly selected person from somewhere else.

So race as an idea is clearly meaningful and there are real, statistically and empirically valid and useful ways to categorize people by ancestry, but in practice, the actual categories in use tend to not be empirically or statistically derived but rather heuristic guesses mapping to the pre-scientifically literate categories we inherited from the first people to travel the world and note that where they went, the people looked different from them. And, of course, the categories in use in practice carry quite a bit of baggage thanks to their use as justifications for slavery, genocide, and various forms of second class treatment and subjugation of ethnic minorities. This makes it hard not to question the motivations of anyone taking race seriously as something we should be using to guide social policy, but also anyone taking it seriously as an object of scientific study at all.

Fair or not, if Scott ever ends up being killed on a hill, this is probably it, as the most obvious difference between his blog and any other vaguely centrist liberal normie like a Matt Yglesias or Noah Smith or Matt Stoller or what have you is that Scott's blog has historically served as a safe harbor for race realists that tend not to be well received almost anywhere else.

Expand full comment

Is a large beanbag a chair? Probably, if you put a gun to my head and forced me to classify whether things are chairs or not, yeah. But it's less useful to talk about it as a chair than otehr things that are more chair-like.

Expand full comment

The concept of "race" is a social construct in the same way that "fruit" is a social construct. In nature, there's no clear demarcation between "fruits" and "vegetables" or "berries"; in fact, there's nothing but molecules, I mean atoms, I mean quantum waveforms, etc. The concept of "fruit" is the description of a cluster that we humans have constructed based on observed properties of objects (which don't really exist, etc.)

However, because these properties are quite real (to the extent that anything is real), it is very likely that humans from different cultures; as well as any aliens and/or machines; would construct their clusters in a very similar way. Yes, "fruit" is a social construct, but that does *not* make it arbitrary.

Expand full comment

So race in the USA could be understood like the indian (hindu?) caste system? Some pre-enlightenment social construct that has little to do with physical reality, but is very important for doing sociology in that society?

Expand full comment

My mom's puerto rican and my dad's a borderer how do I answer race?

Expand full comment

I flipped a coin and answered Hispanic.

Expand full comment

If you got pulled over by a platonic Racist Southern Sheriff, would he use a slur and if so, what slur would he use?

That's your race.

Expand full comment

Doesn't always work. I know a guy who's half Chinese and half Italian, and has been called a spic, and even the n-word.

Expand full comment

That's happened to me roughly 3 times

Called a cracker by some young gangbanger, a Gringo by a different one, and a Beaner by a group of Nazis

So IDK

Expand full comment

Survey 23, "Metaphysics" leaks the survey results after submission, meaning you can just go and look at everyone's answers, including their identifier. https://i.imgur.com/Gdo8Sa3.png

This seems like a bad thing.

Expand full comment

Why is it a bad thing?

Expand full comment

Metaphysics in particular is probably fine. If it was the transAGP one or other sensitive ones, and somehow the demographic sheet almost got leaked, manual de anonymization. I suspect this isn’t a big issue

Expand full comment

A person of ill intent could corrupt the data by using someone else's identifier

Expand full comment

Oh, good point.

Expand full comment

My quibble on the demographics survey: it treats "having a mental illness" and "not having a mental illness" as a binary. What if I was diagnosed with the condition in the past, but no longer have it?

Expand full comment

Completed all that apply to me, except 21 which is broken as others have reported already. Officially bragging.

"Metaphysics" was very fun to do, although the two math problems seem exactly equally hard to me.

"Understand your own beliefs" was also nice because I was surprised to actually feel represented by some answers. Spotted some conflict-vs-mistake-theory in there :)

The one about weight gain / loss left me somewhat confused. Not much happened directly before my birthday in 2018, and I had no idea why it was relevant.

"Health-Related Quality Of Life" was not explained very well (too wordy and therefore more confusing that it needs to be), but the research topic is super interesting and I hope something relevant comes out of it. I had to concentrate a lot while answering, though.

Expand full comment

Oh, and in the COVID-19 Impact, it looks like my income went down badly in the pandemic, but that is just because I quit my job to do something I liked more that pays less. *shrug* There could have been an option or direct question about COVID-related loss of employment.

Expand full comment

Hey! Thanks for the feedback. Don't worry, it's actually going to be disambiguated based on a previous question (IE, did you experience loss of employment/work less hours/take a pay cut due to COVID).

Expand full comment

Agreed on a lot. The metaphysics one was hard.

Also on the weight gain/loss, I put on about 20 lbs this past year sitting around during Covid, so I dunno, I guess that counts. I lost about 20 lbs once after an adult tonsillectomy, so I'm not sure that counts, since it's obvious WHY I lost weight. But I did my best.

Expand full comment

Losing weight after getting your tonsils out would count under the hospitalisation option (there's a medical reason as to why you lost weight).

I lost 20+ lbs in eight months after going on diet/exercise, then got whammied by the swine flu and a very severe weather period where there was no going outside but "stay huddled over a heat source indoors", so that knocked me off both diet and exercise regime and I put the weight back on.

There should have been a yo-yo dieting question included, because I've done the "gone on diet/lost weight/came off diet/put it all back on and a bit extra" more than once.

Expand full comment

The math problems seemed equally _easy_ to me, but I was started to recognize how much I preferred working with the fractional representation.

Expand full comment

Interesting. I would have said the fractional one is easier, but my intuition much prefers working with the other one, because when my brain is turned off I try to compute values and multiply them, which the fractions make annoying. Doing the "clever"/methodical thing means being awake, which is more work to me!

Expand full comment

If they were five digit numbers I’d kinda get it, but how are people seeing any sort of difference between two second grade math problems lol

Expand full comment

The right math problem gave me traumatic flashbacks to formal logic in uni. Left any day.

Expand full comment

1.5 x 2 gave you formal logic flashbacks?

Expand full comment

..... oh god, now I understand what it was supposed to be. I completely visually misidentified it as a natural deduction term. Which makes no sense, but I immediately stopped thinking about it so...

Expand full comment

Apparently horizontal lines and free variables (I read "x" as a free variable) are just triggering to me now.

Expand full comment

I didn’t get the math problem one at all. Maybe a preference between decimal and fraction? I don’t think anyone cares about that tbh

Expand full comment
founding

Someone commented elsewhere in this thread with a corrected link to survey 21.

Expand full comment
founding

* elsewhere in the comments on this post

Expand full comment

Yes, did it. Thanks :)

Expand full comment

Can't load 21. Error: "A problem occurred:

Couldn't find cookie for study result ID 31188. Are cookies allowed in your browser?"

Expand full comment
founding

Someone commented elsewhere on this post with a corrected link.

Expand full comment

Survey 9 doesn’t work. After submitting a few pages of responses, one page doesn’t let me submit, instead telling me “ The question has changed. Please review your response.” This happened several times.

I took other surveys and they worked fine.

Expand full comment

(ROT13'ed; please don't read unless you have completed or don't plan to do the digital literacy survey.)

Ner Cebklcbq naq Svgvoyl va gur qvtvgny yvgrenpl fheirl vagragvbanyyl zrnavatyrff grezf gb grfg jurgure gur ernqre vf nafjrevat ubarfgyl?

Expand full comment

That was my assumption.

Expand full comment

That's what I thought too.

Expand full comment

Same. Though I assumed I just hadn't heard of one.

Expand full comment

Yeah, same. Didn’t even have to derot13 it to know what meant lmao

Expand full comment

For #0 on the psychiatric conditions I wish there was an option for 'I have been formally diagnosed, but have since recovered"

Expand full comment

Oh, that would be a good survey. What people have been diagnosed with and then a few years later if it changed/what it changed to/if any treatment or interventions were used.

Expand full comment

Another vote for Survey #21 not working - Chrome on Windows.

"A problem occurred:

Couldn't find cookie for study result ID 31188. Are cookies allowed in your browser?"

Yes, cookies are allowed in my browser. No, I was not expecting a study result prior to submitting some information to the survey.

Expand full comment

Chiming in yet again to say that “country that you most identify with” means nothing, and that you should split it into “country that you currently live in” and “country where your primary cultural background is from”, or something similar. I will continue to complain about this on every survey that has this question.

Also seconding everyone who complains about the race question. I already hate the US system of racial divisions, but if you’re going to use it, the bare minimum should be to allow for multiple selections and to have a place where you can write-in an “other” option.

ok enough griping time to fill in two dozen surveys

Expand full comment

I'm concerned by this as well, "identify with" is not something I'd say about any country at all. What do I put in to not get my answers filtered out?

Expand full comment

Yeah, that country question was weird. I just answered with the country I live in, though I could have answered with the country I belong to culturally or the country I belong to racially instead.

Expand full comment

People may identify more with a particular country other than the one they live in, e.g. if you've moved to another country for work for a few years, you probably identify with your country of origin more?

I can see that "my granny came from Taiwan but I was born and raised in Fresno" does have that cultural/domicile split you mean, but then again - a lot of Irish-Americans identify very strongly with Ireland, even if they were born and raised in the US and their connection is a great-grandparent.

Expand full comment

I like these comments because they challenge a lot of my (American) assumptions. I've always felt like an American, and can't really imagine feeling, say, Australian or Canadian, even if I lived there for many years, let alone a country where I might feel more like a foreigner.

Now, asking what US state I'm from can be a harder question; I guess for you (Kronopath) a country must feel more like a US state does to me? Or is not being "from" a place more of a thing outside the US?

Expand full comment

Agreed. It should be split into "Country of birth"/"Country of residence" or similar. If anything, I feel much more country-fluid than gender fluid

Expand full comment

How am I supposed to procrastinate on this if you don't give us a deadline?

Expand full comment
author

I'll probably close this sometime around the end of July.

Expand full comment

1. What should atheist Jews put for the religion question?

2. If you think I might have a psychiatric condition (no diagnosis) and also have family members with the condition, which should I check?

Expand full comment
author

1. If Judaism is important to you, Judaism. If you're ethnically Jewish but don't believe any of it, atheist.

2. B

Expand full comment

>Automation and technology is going to rapidly take away many people's jobs.

I always hate just answering with either agree/disagree on questions like this. I feel like most people's interpretation if I agree will be that it's bad and might generate a headline like "85% of economist are worried that automation is going to take away people's jobs."

How I want to answer that question is "Yes they will and that's a good thing. Here's why...."

Expand full comment

Yeah, same here. I hope it brings true post-scarcity society.

Expand full comment

I've never understood why some people believe we're suffering from a lack of stuff. It doesn't get any more post-scarcity than this. If people have found a way to keep the urgency flowing in this environment, they will do so indefinitely.

Expand full comment

Oh, I agree with that mostly. GENERALLY, for most people in the world, you don't go hungry, you have the internet, and you have reasonable access to medical help. (Not to forget the 1 billion or so for whom this does not yet apply, but it is getting better.)

The "stuff" part is just a part of it, but I hope to see less scarcity of freedom to travel, space to move around in, and time to do meaningful things. Robots will help, in theory. Especially when they're building huge orbital cylinders in the sky that are actually nicer than Earth, and where you can go surfing in the morning and then skiing in the afternoon, and you also have a hummingbird sanctuary.

"Post-scarcity" means big things to me, but I'm hopeful it happens someday.

Expand full comment

That's kind of my point: the lack of space and time is mostly self-inflicted. There's plenty of space--people just don't want to live there, because that's not where the party is. You do have to work a job, but you could also work a lot less if you lived in a place with more space.

We generally push ourselves to the limit, because status (and the next best thing) is always scarce. I say this as someone who fantasizes about more space and free time, and complains about our economic ponzi scheme, but hasn't so far taken any steps to move to a cheap off-grid location and work part-time for the local town.

Expand full comment

"where you can go surfing in the morning and then skiing in the afternoon, and you also have a hummingbird sanctuary"

That's not going to happen, or if it does, not for the ordinary guy. The upper-middle class? The professional class? The college-educated going into a STEM job class? Sure.

Joe on the warehouse floor or Jill working in the hair salon? No.

Expand full comment

Why not? The huge orbital cylinder will need warehouses and hair salons.

Hot water that nobody else has used yet used to be an expensive luxury. Presumably some of today's expensive luxuries will become common in the future.

Expand full comment

I hope that someday people will look back at the consumption possibility frontier of today's billionaires and decide they'd rather be the poorest person alive then have to live like early 21st century billionaires. Kind of like how most people today would rather have the consumption of a normal middle-class American then a medieval king.

(And that we'll do all that with even less harm to the planet)

Expand full comment

When I hear people complain about inequality today and the wealth gap between the poor and the billionaires I don't think "yah, lets take away the billionaire's wealth" I think "Wow, we've got to find a way to get everyone that type of wealth."

Expand full comment

I don't necessarily think "we should take all that money away" but I do think "if you are making money literally faster than you can give it away, then maybe how about we re-design things so that you get a decent but not decadent level of return on your entrepreneurship, and the workers get better pay and conditions and don't have to wear incontinence diapers on the shopfloor?"

https://www.npr.org/2021/06/15/1006829212/mackenzie-scott-is-giving-away-another-2-7-billion-to-286-organizations

"Saying that she's troubled by the increasing concentration of wealth, philanthropist MacKenzie Scott says she is giving away another $2.7 billion of her fortune to 286 nonprofit organizations.

Scott, who divorced from Amazon founder Jeff Bezos in 2019, made the announcement in a blog post on the website Medium. She has so far given away more than $8 billion in three rounds of funding revealed in the same manner. Her net worth is estimated to be nearly $60 billion.

In her divorce from Bezos, Scott received a 4% stake in Amazon. But shares of the company's stock rose sharply during the pandemic, and despite giving away billions, Scott's wealth keeps growing. Her net worth, estimated at $36.1 billion in October 2019, has increased by some $23 billion since then."

I don't mind Jeff Bezos having five or ten billion due to the success of his company. But tens of billions that keep on growing with no input, while at the same time Amazon runs a campaign to make sure that the workers in a particular region don't unionise (which is their right to do?) *Something* is out of balance somewhere.

Expand full comment

I think the thing that's primarily out of balance is the moral compass of any billionaire who isn't actively working on efficiently giving away their money until they're not a billionaire. I don't fault MacKenzie Scott or Bill Gates for being slow and deliberate in their choices of which organizations to donate to. And I think that Gates-esque billionaire philanthropy is likely to do much more good in the world than an equivalent amount spent by any major government. But, if a government is able to do net good with its spending, I think highly progressive taxation is a great way to deal with those non-altruistic billionaires.

Expand full comment

I would really like automation and technology to rapidly take away many people's jobs, but I feel like politicians might just make *more* awful jobs for people to have!

Expand full comment

Why would you point the finger to politicians? They're not the ones employing people who aren't given bathroom breaks and whose continued employment depends on the whims of an algorithm.

Expand full comment

Politicians make the laws and set up the programs that demand that people work, legislate automation, and so on. I feel that Amazon and the like would be happy to automate their workers away (and I would be happy for them to do that as well); it's governments (and a society polluted by outdated rhetoric that's too focused on employment) that do the things that Scott discusses in this article: https://slatestarcodex.com/2013/12/08/a-something-sort-of-like-left-libertarianism-ist-manifesto/

Even during the pandemic, much mainstream rhetoric has focused on incentivizing people via policy to go back to work — even though it's not safe for many of them to do so. Politicians and society together will probably continue to insist that benefits be reduced to push people back to work, thus ensuring the suffering of employees who don't actually need to suffer at all.

Expand full comment

Yeah, it'd be lovely if all the low-tier jobs were automated away and people were able to get money to live somehow, but that's not going to happen.

Amazon and others will, as you say, be all too happy to automate these jobs so they don't have to employ people. But that does not mean that the former warehouse workers will now all be free to do better jobs - because unless they upskill themselves, those 'better jobs' don't want them - this is the "coal miners become computer programmers" idea.

Governments will not provide social welfare to people on the grounds that "there are no jobs out there". So you either work whatever shitty jobs are left, or if you can manage to jump through the hoops of training programmes hope you can find a decent job, or you patch together some combination of "do cash-only black-market jobs, social welfare benefits, and borrowing from family" to keep you going.

Post-scarcity society won't mean "everyone has UBI and can be creative in their free time or decide to work a part-time job", but it will mean "billionaire shareholders in the big companies get even richer as the value of their stock holdings increases", so Jeff Bezos can play around with his very own toy rocketship set!

Expand full comment

While I do completely agree that politicians ought to implement some sort of unconditional UBI, I wouldn't frame their failure to do so as "demanding" that people work. I admit from a utilitarian perspective it's a distinction without a difference, but well, most people aren't utilitarians.

Expand full comment

Also, I'm not clear on whether 'take away jobs' is saying 'unemployment will go up due to automation' or 'people will move to new jobs due to automation.'

Expand full comment

Re customer satisfaction survey --- I read ACX via the RSS feed. So if the email letter stopped I would keep reading it but I wouldn't go to the website.

Expand full comment

On the demographic survey, it should probably ask how long people have been reading ACX, not SSC - or maybe mention both. But some new ACX readers might not even know what SSC is.

Expand full comment
author

Thanks for catching that! Fixed!

Expand full comment

But now it asks only about ACX, so it's impossible to truthfully answer anything other than ~6 months ago or less. I'd say it would be better to include both.

Expand full comment

Your fix confuses me. Should I answer 6mo now to "How long have you been reading ACX?"? Or should I answer "More the two years" because I read the predecessor to ACX? And how would I know without coming to the comments section like I just did?

Expand full comment

I came to the comment section precisely to ask if the 1+ year options were "is Barack Obama a hippopotamus"-type control question, just subtler.

Knowing the previous wording mentioned SSC means that that's not the case, but it was definitely confusing to my pedantic self.

Expand full comment

But damn it, now this is like the Mahabharata! How do I *know* that someone somewhere (maybe a zookeeper or park ranger) *hasn't* named a hippo "Barack Obama"?

" Krishna knows that it was not possible to defeat an armed Drona. So, Krishna suggests to Yudhishthira and the other Pandavas, if Drona were convinced that his son was killed on the battlefield, then his grief would leave him vulnerable to attack.

Krishna hatches a plan for Bhima to kill an elephant by the name Ashwatthama while claiming to Drona it was Drona's son who was dead. Ultimately, the gambit works (though the details of it vary depending on the version of the Mahabharata), and Dhristadyumna beheads the grieving sage."

Expand full comment

I guess maybe I should have skipped the psychadelic survey entirely, but it was weird that there's no way to answer "I tried one and nothing happened".

Expand full comment

I frequently find myself confused by the questions in the "Metaphysics" survey — sometimes none of the possible answers seem correct at all, or else the question is based on a false premise. Oh well; guess I'll just try to pick the least incorrect option, and skip the worst offenders. Hopefully the results aren't announced without describing what the options were.

Expand full comment

I can't tell if "how long have you been reading ACX?" Is supposed to be a trick question. The first post was (just) less than a year ago, but I've been reading Scott's writing for at least a decade, so I assume I'm supposed to answer more than 2 years?

Expand full comment

This also confused me. According to another comment, the question was originally about SSC only, and someone pointed out this could be confusing for new ACX readers who don't know about SSC. Scott changed it to ACX, but I assume the intent is still to take SSC into account.

Expand full comment

I likewise wondered about that. I answered it as "less than 6 months" because I thought it might be a question to filter out liars.

Expand full comment

The weight gain/loss survey does not ask about pregnancy, and there was no comment section. This seems like a significant oversight.

Expand full comment

Agree, it needs a comment section.

Expand full comment

I did them all. But I didn't read the first survey clearly and failed to get a proper user ID for half of them. Do I still get the praise?

Expand full comment

When I try to load 21, I get: "A problem occurred:

Couldn't find cookie for study result ID 31188. Are cookies allowed in your browser?"

Expand full comment

Maybe sometimes it looks like 81133.

Expand full comment
founding

Someone commented elsewhere on this post with a corrected link.

Expand full comment

> When you dream, does it occasionally happen that the *you* is a different person from your waking self? If so, is that person male or female?

I'd say in the last 5 years I've had a dream that I remembered upon waking 10 times, and even at that never to the level of specificity that would allow me to answer these questions.

Expand full comment

But it provided a box you could check for exactly that.

Expand full comment

I assume you mean the "never happens" box? I believe that means that it never happens that I dream that I'm someone else. I do not know if that happens, so I don't think you're correct.

Expand full comment

There was a "The sex is never / almost never identifiable" option, although I guess that is technically different from the sex being identifiable and you just not remembering it.

Expand full comment

I answered everything I could and it was fun! A few comments:

- Americans (as in the USA, but "people from the USA" is a mouthful) obsession with race is downright bizarre coming from a European point of view. Maybe a survey about that would be interesting?

- I've seen the word "community" used and thankfully it was explained (a city, town). I often hear it, and feel like it's also a very American point of view. For me, a community would be people you are close to, share something with. Family and friends mostly, maybe a few colleagues? But the question seems to imply that it could be two million people. Does that word has a strict meaning?

- Behavior during COVID pandemic: some questions were necessary to answer while a question before essentially made them moot, so I had to reply with bad data.

- Weight gain/loss: no mention of the COVID pandemic? That seems like a big oversight.

- 1 to 5/1 to 7/strongly disagree to strongly agree questions in general: I'd like a metasurvey about this. I have a hard time determining if I feel more like a 5 or a 6 on most issues. Does "strongly disagree" means "equivalent to the thing with you disagree the most", or "equivalent with the thing that you saw people disagree the most"? For example, I think most people are more vindicative about issues than me. Does that mean that I should only answer in a reduced scale, or should I use the full scale. Are there studies in this area to determine if this kind of question is meaningful even if it's only relative? Is it a case of the perfect being an enemy of the good?

- again a metasurvey proposition: I might be the exception in this community, but I don't feel strongly about most moral issues. Do we have any data on that?

- Understanding your beliefs: Is it bad to be a homosexual: I don't know if it's on purpose, but it's confusing to answer. I don't have anything against homosexuals in general, but I've endured some suffering because I was one that non-homosexuals haven't endured.

Expand full comment

Hey! Covid author here. What do you mean by "a question before essentially made them moot, so I had to reply with bad data"? It might be something I can fix (or might not actually matter).

Expand full comment

"If you at any time have felt that the pandemic was easing up, please check all the reasons that apply", which is a required answer with no option if you didn't feel that the pandemic was easing up, "If any of the following influenced your perception that the pandemic was easing up or ending, please check all that apply:", same thing, "Which has been the single most impactful influence on your perception that the pandemic is easing up (if you have felt that way)?", same thing too.

Expand full comment

Ahhh, crap. That was the other COVID survey. Alas I do not have any influence.

Expand full comment

Hi, I wrote that one. Very good point. This is the first survey I have written, I did it under time pressure, and definitely did not think through some of the implications of the way it was structured. I appreciate the feedback. Partly it was that there were so many possible reasons, I was not sure how to structure a question with thirty possible answers. It might have been better if it were one question. And you are right, the assumption was built into it that someone would think it was "easing up" which should not be built into the question. I appreciate the feedback. At some point I would like to do a more solid version of it, if I ever get an audience opportunity.

Also partly those questions were casting a wide net so I could find out which reasons other people thought were important. I think there is a lot of terrain there to cover.

Expand full comment

I appreciate the response! As long as you're conscious of it, you'll be able to exclude the bad data based on other answers so I think it's okay. Google forms are also not perfect and sometimes lack some subtlety. If you do another study, I'll be happy to answer it.

Expand full comment

I couldn't answer the questions about which things worried you more (choose 1) because those things didn't worry me at all.

Similarly >Have you at any point since February 2020 felt that the pandemic was easing up/not as bad as it used to be?

I don't really think that pandemic during its entirety was as bad as it was portrayed by the media and panicked people. Reaction to the pandemic was much worse than the pandemic itself.

I didn't fill the questionnaire.

Expand full comment

On the Covid-19 Impact survey, for the question "During the COVID-19 pandemic, did your employment require in-person interaction with the general public?", it doesn't give the option to indicate that it's not applicable because I was not employed. I selected "unsure / prefer not to say", but I see that the survey creator is active in this comment section, so perhaps they can give me some guidance.

Expand full comment

You did right! But just for clarity, I added a not applicable option.

Expand full comment

Thanks :)

Expand full comment

My answers to some of the sex questions are completely dependent upon how in on it the other person is. Like, touching a stranger at a sex club where everyone there is like "yeah, this is where we will get touched by strangers" is completely different from touching a stranger on the bus.

Expand full comment

Yeah, and hurting or getting hurt by your partner is different depending on whether that's the kind of sex you've agreed to.

Expand full comment

I've been answering under the implied context of “assuming an appropriate context”, since the contrary distinguishing state seems clearly to be “would not be interested/aroused regardless of how it were set up”.

Expand full comment

I literally don't fall into any of the political buckets. Normally I'd say this is a matter of "I'm so special" syndrome but when you can't agree with at least half of any of the combined characteristics, what are you to do?

Expand full comment

Father of 11? I’d say you are pretty special in 2021. Fathers Day must be a wild time.

Expand full comment

4 teenaged boys with # 5 turning 13 in 2022. Pray for me.

Expand full comment

Small point, but in Survey 0, I interpreted the question "How long have you been reading ACX?" to include SSC as well. I assume this is what was meant, but some people might take it literally.

Expand full comment

I'm sure you're inundated right now with people asking questions about the survey, but are the Life/Job/Social/et-al. satisfaction sections gone? Those results always seemed the most interesting to me.

Like, lawyers reporting higher average life satisfaction than software engineers was an unexpected and interesting result that kind of cut against the voguish archetypes of the unhappy, stressed lawyer and the chill, relaxed software engineer.

If they are gone, maybe they can make a comeback in future iterations?

Expand full comment

Completed all of them, except for Optical Illusions, which did not load in the time it took to take all other surveys.

Expand full comment

You´re a hero! And if you want to wipe out that flaw of one missing study, try the fixed link ;)

https://jatos.mindprobe.eu/publix/1521/start?batchId=1702&generalMultiple

Expand full comment

Filled out most of these surveys, but the only one that caught my interest was survey 23, metaphysics. I would love to see the data on correlations between answers to those questions, but there is no information of who the survey is made by, or what the data will be used for.

Expand full comment

I made it, specifically to look at the correlations between answers to those questions. I'd like to release the data (but I have no way now to retroactively get explicit permission, so I may have to remove the early answers), and also to blog about the results (maybe on LessWrong).

Expand full comment

I got an email saying "GreatSwordsmith liked your comment", but I don't see any way to "like" comments, nor any indication of comments having likes. Is that visible only to paid subscribers?

Expand full comment

Oop, I broke substack. When I 'liked' your comment, I did so by pressing a button in the email I received notifying me about your reply. Turning off the 'liking' feature on his blog was one of many changes to the comments system Scott asked substack to make when he joined, but it looks like you can still do it from emails.

Expand full comment

You can also "like" things if you have the ACX tweaks chrome extension

Expand full comment

22 More Psychedelics the questions about health insurance are less meaningful if you live in a country where that's not the main way people get health care. (As is the case for much of Europe). I answered as about whether whoever the healthcare provider was should provide it which I assume was the intent

Expand full comment

I'll say that even as an American, my answer to that question had a lot more to do with the dysfunction of American healthcare and insurance than it did with the therapeutic legitimacy of psychedelics.

Expand full comment

I just finished all except 2 that didn't apply to me and the Optical Illusion that doesn't work.

Expand full comment

And about race, many fall under Other I guess. There's no Mediterranean, there's South Asia but it doesn't say it includes all Indo-Aryans (thus getting rid of the vague variety box of Middle Eastern and put Semitic instead), there's no Central Asian, and I doubt you Americans would count the majority of real Caucasians as Caucasians if you saw one.

Expand full comment

Sorry, there was apparantly some mix-up of the links for study 21.

This one should work, if you are still motivated:

https://jatos.mindprobe.eu/publix/1521/start?batchId=1702&generalMultiple

Expand full comment

Now it's daytime and bright, I'll do this tonight.

Expand full comment

In "behaviour during covid" the question "If any of the following influenced your perception that the pandemic was easing up or ending, please check all that apply: *" seems to list a bunch of things about my personal experience and feelings. Which might effect how much I follow the lockdown rules, but are a separate thing from whether I feel it is easing up or ending

Expand full comment

Hi, I wrote that one. You are right, part of my goal with that survey was to sort out the attributes of that connection, between how someone feels about it or thinks about it and what they do. This community probably more than many others really will "follow the data" in terms of seeking it out and adapting their behavior to it to get the results they want. But humanity in general, how people feel about something will alter how they think about it and what they do. There can be a feedback loop, "my kids are miserable" (for example) -> "I wish the pandemic wasn't so bad" -> "therefore the pandemic isn't so bad anymore." Magical thinking, wishful thinking, motivated reasoning. I did not really know how to start asking about that in Covid terms and so cast a wide net. Should it be one question, not really, but I didn't think of a way to group the possibilities meaningfully into multiple questions and if I had I think the survey would have been too long. I appreciate all the feedback and in addition to looking at the data I will also look at ways to revise the survey in case another opportunity for an audience comes up in some context. There definitely should have been more possibilities as well.

If you have better ways to phrase questions let me know!

Expand full comment

I can see the motivation for asking that, but I don't think people's self report is going to be accurate. E.g. an external observer will maybe be able to observe "Bob stopped taking pandemic restrictions so seriously because he was sick of being at home all day" but Bob would probably self report that he had made a decision based on the data.

A better method might be to ask people to separately report how they felt at different times and their assessments of the pandemic at different times, and see if there's a correlation.

Expand full comment

I did 23 of them. Didn't do the optical illusions because I got the same error as many below. Probably not how I should have spent my day.

Expand full comment

Sorry, there was apparantly some mix-up of the links for study 21.

This one should work, if you are still motivated:

https://jatos.mindprobe.eu/publix/1521/start?batchId=1702&generalMultiple

Expand full comment

In the "can society be wrong about words" question on the metaphysics survey, the question sheet only showed me two options but the survey results page shows four. Also, that was the only survey (that I took) which had a "see previous results" link at all after submitting – was that intentional?

Expand full comment

My fault. I was editing the wording of some questions when Scott published the survey. When you change the wording of an answer on a released survey, previous answers are listed separately under the earlier wording.

Expand full comment

Letting people see previous results was intentional, but someone wrote above that it was bad. I... don't understand why? The survey is anonymous, even with the user ID. I've switched viewing results off for now, but would like to make them public.

Expand full comment
founding

why does the weight survey not ask about having children?

Expand full comment

I wondered that as well

Expand full comment

I concur, that was definitely the defining factor for me and I was disappointed it wasn't listed.

Expand full comment

Pretty sure survey author is male.

Expand full comment
founding

I concur, I'm a male and taking care of a baby added enough stress to my life that I was eating a lot more.

Expand full comment

Regarding #14, businesses being "larger than some countries" isn't saying much in a world containing Vatican City and Andorra and San Marino. Yes, I'm totally fine with a company being larger than Andorra.

Expand full comment

Yup, badly worded. I decided to interpret it as instead meaning "typical countries that the author might be thinking of".

Expand full comment

#6 let me submit a response even though I didn't click an answer for the second question... oops.

Expand full comment
founding

The rhyming survey is labeled as "targeted at non-native English speakers", but the survey doesn't actually seem written with that assumption. "English" is an option for native language, and "US English" is given as an example on the dialect question. It looks more like "we're studying how non-native speakers differ from native ones" which means that responses from native speakers are needed in order to have a control group. The label may be discouraging some native speakers from usefully participating.

Expand full comment

At the end of the cryonics survey it says "Please check reddit.com/r/cryonics in January 2020 for survey response data. " So unless the cryonicists are branching out into time travel I think there's a typo

Expand full comment

Some surveys don't ask for the ACX ID; mentioning in case this is not intentional. (Sorry, did not keep track which, was focussed on going through all of the surveys.) I added it in a freeform comment field for one of them, but that wasn't always possible.

I just reused the SSC ID I gave myself in a previous reader survey a few years ago, by the way. I'd saved it in my password manager and there was no reason not to reuse it, really.

Thanks for the surveys, everyone who made them! :)

Expand full comment

Scott: It wasn't clear to me how to record a historical rather than a current diagnosis in the mental health questions in the demographic survey. I can see in the comments other people had the same issue. Maybe change the wording to "I currently have or have had a diagnosis" next year?

Other survey writers: please understand that not everybody is from the US, and phrasing your questions assuming everybody is American makes them hard for us to answer, affects your data quality, and also drives many of us slightly insane. For example, if you want to know my opinion on right wing people, that isn't the same as my opinion on American Republicans.

Expand full comment

I had the converse problem with a survey question that assumed everyone uses the metric system. We Americans don't know what "30 cm" is without Googling (it's about 11 inches.)

Expand full comment

Unless I missed it, incest survey did not ask for unique ID.

Expand full comment

Yeah, and it reasked some of the general survey questions.

Expand full comment

For #2, i assume i should exclude stepbrothers and half-brothers with a shared father?

Expand full comment

I wrote this, and this was an underspecification on my part. I meant full brothers, or brothers with a shared mother. Stepbrothers should be excluded.

Expand full comment

The Optical Illusion survey doesn't load for me (gives me an error about not being able to find the study result Id).

Expand full comment

Likewise for me.

Expand full comment

Sorry, there was apparantly some mix-up of the links for study 21.

This one should work, if you are still motivated:

https://jatos.mindprobe.eu/publix/1521/start?batchId=1702&generalMultiple

Expand full comment

Did 'em all other than the non-functioning optical illusion one and those (if any) that were demonstrably intended only for demographics excluding my own. The metaphysics one was repeatedly infuriating, demanding that I choose between two options neither of which I could honestly endorse.

Expand full comment

The metaphysics survey *was* annoying, but I like the ideas it discussed and questions it raised :')

Expand full comment

Yeah, my submission for the metaphysics one ended up with a number left blank, as both the options would be things I somewhat aggressively disagree with.

Expand full comment

Sorry, there was apparantly some mix-up of the links for study 21.

This one should work, if you are still motivated:

https://jatos.mindprobe.eu/publix/1521/start?batchId=1702&generalMultiple

Expand full comment

Survey #3: What do the colors mean? There's red, yellow, green, and blue, which are supposed to represent some of different health status, but none of those are mentioned in the survey text.

Expand full comment

The colors aren't saying anything the words aren't saying. It looks like green is used for "perfect health", yellow for moderate problems, red for severe problems, and blue for the various "enhanced" categories.

Expand full comment

Scott, there was no place to insert my random number identity in the Incest survey.

Expand full comment

I guess that means that that survey author doesn’t want the demographic information?

Expand full comment

I never understood why people fill in nonsense on surveys, until I got to the option "An unrelated man and woman: sexual relations are illegal."

So tempting...!

Expand full comment

Finished the survey! More next year please.

Expand full comment

I found the "Personality" questions extremely frustrating - was it intentional that many (most!) of the questions had contradictory descriptions?

I see "humble" and "unselfish" as entirely separate and unrelated traits.

Same for "Interested in art, deep", "Amoral, carefree", "Ethical, unforgiving", "Principled, uptight", and "Imaginative, odd".

A couple of those pairs are close to opposites.

Expand full comment

Question about how to answer the "work status" question on the reader survey: There is an "Academics (teaching side)" But there is no "Academics (research) side"

I work in a university lab doing research. I do literally zero teaching.

My best guess is either non-profit, or "governmental work" since all of our funding is state or federal grants.

Expand full comment

I am in the same situation. I was wondering if "Academics (teaching side)" could be interpreted as being academic staff rather than a student.

Expand full comment

Dos anorexia proper - not anorexia nervosa, but "accidentally loses weight due to insufficient appetite" - count as an eating disorder?

Expand full comment

Appetite loss due to an underlying medical condition has a diagnostic code in the ICD-10 system, but would not be considered a mental illness, if that makes sense. But if the appetite loss seems to be related to say generalized anxiety or another mental disorder but doesn't meet the criteria for nervosa, then it might be categorized as an eating disorder, not otherwise specified.

Expand full comment

It's not so much "appetite loss". It's not new; I was horrifically underweight all the way through school (I think I hit 150 cm while weighing 32 kg).

I ended up putting "no" because it's not tied into the sorts of pathologies seen in AN and bulimia.

Expand full comment

The "neither does anyone in my family" for the mental health questions is ambiguous. Do you mean genetic family (ie you're interested in the genetic connection & no one else counts) or family in a broader sense (in-laws can give you social experience with people who have the condition but won't pass it on genetically).

Expand full comment

I interpreted that as genetic relations only, and the "within two generations" part as anyone who was equally related as a grandparent or grandchild (25%)

Expand full comment

Yeah, that's how I answered it, but I thought it could be clarified.

Expand full comment

As a point of self-comparison, I seem to be getting half a point more politically right-wing each survey. I wonder if this'll keep going, and (if it does) if I'll move far enough to actually vote for a right-wing party by the time I'm actually able to vote somewhere.

Expand full comment

Reading SSC and DSL has made me more conservative over time, though still not enough to make me ever consider voting Republican.

Expand full comment

What's DSL?

Expand full comment

Data Secrets Lox - it's a forum for the SSC exodus, and where a lot of the ring wing commenters ended up.

Expand full comment

I was completely unable to load the Optical Illusion survey. Error message "couldn't find cookie result for study result ID 31188. Are cookies allowed in your browser?" Yes, yes they are. Am I trapped in a metaphysical anomaly?

Expand full comment

It's a known issue, been reported hours ago. That survey is just borked.

Scott should probably put a comment in that it's a known issue.

Expand full comment

Sorry, there was apparantly some mix-up of the links for study 21.

This one should work, if you are still motivated:

https://jatos.mindprobe.eu/publix/1521/start?batchId=1702&generalMultiple

Expand full comment

I'm in! Thanks.

Expand full comment

Well I tried, but could you please put a progress indicator on it? I gave up after ten minutes of frequently repeating patterns.

Expand full comment

Yeah, sorry for that, one actually needs a LOT of trials in perceptual experiemnts like these (already reduced the number from about 200 trials that would be optimal to about 70 that we are doing now. But with 10 minutes you were close to the end I think and we can use the data, so thank you very much for participating!

PS: Working on the progress bar though. I am used to having participants in the lab, where they can´t get away. So we are not really used to focus on user friendlieness I guess...

Expand full comment

At ten minutes, I also gave up. I was not expecting it to take that long, and the lack of a progress bar meant that I had no way of knowing that I was close to being finished. Sorry for not sticking through it to the end.

Expand full comment

No problem, we can use the data you have generated untill then. Thanks a lot for participating!

Expand full comment

It took me more than 20 minutes to finish the illusion part, and I'm afraid I might have convinced myself that some images were uniform just to be able to be done faster (although I tried really hard to not do this). Furthermore, I managed to avoid intentionally looking away from the red cross, but I spent the whole time intensely curious about what the image actually was, and I really wish there had been the opportunity to play with the illusions at the end.

Expand full comment

Regarding the Survey on feelings of gender:

"To what extent do you find yourself behaving differently when in a group of all/mostly women compared to when in a group of all/mostly men (which are otherwise equivalent to you)?"

I don't know, I haven't been in such a group in aeons, unless you count the internet, which makes gender harder to track.

"You wake up one morning with your body changed dramatically in one way, but otherwise as healthy as before. Which bodily change would feel the strangest/most disruptive to your perception of self?

30 cm taller than you were before

With the opposite sex organs (male to female and vice versa)

With a drastically different skin color (e.g. black to white and vice versa)"

Is this about swapping primary sex characteristics only, such that, say, a cisgender person would wake up with a body comparable to a transgender person of their gender who was unusually lucky with HRT results but had no surgical interventions, or is this such that a cisgender person would wake up with a body comparable to a cisgender person of the opposite sex? I am tentatively assuming the latter, but the phrasing is ambiguous.

Expand full comment

I interpreted the "opposite sex organs" as switching the genitals only (so for example a post-op trans person would have their genitals reverted to initial state), I agree it's confusing.

Expand full comment

Finished all the surveys that pertain to me, other than 3, as it was asking me to imagine things I cannot imagine, and to evaluate them in a framework which is utterly nonsensical to me. Answering surveys can be quite fun, it seems!

Expand full comment

congrats!

Expand full comment

Since 31 is a redirect to 19, if we finish 31, should we do 1 or 20?

Expand full comment

The way the 24 - 31 options are spaced out, I figured the idea was to continue at 20.

Expand full comment

I think the "intellect significantly enhanced" option in 3, while a decent specification for an ordinary sample, was not a great choice for ACX; it specified as IQ 140-150 and IIRC the average IQ here is nearly that, so it's going to return a whole pile of "no use" and "negative use" results.

Expand full comment

The average IQ here was that last time we checked. I'm curious if that changed with recent very public events.

Expand full comment

Nah, mean and median of the 2020 survey were 137.4 and 136 respectively (disregarding the people who said "0" and "1e40").

Expand full comment

Also, that's of the 25% of the survey-takers who actually gave an answer. I'm suspecting that high-IQ people are more likely to take an IQ test than average-IQ people, so that's probably an overestimate.

Expand full comment

I thought that as well. That both high and low IQ people, along with those with a gap between IQ and school performance, are more likely to be tested. I didn't answer, because I never took an IQ test, because I seemed close enough to average it never came up. It seems like this might be common.

Expand full comment

There was a post on SSC a while back digging into that. There is no such reason not to specify your SAT if you live in the US, but SAT results matched the IQ results. (There's a known relation.)

Expand full comment

I had heard that, and I think the other SSC survey asked about that as well. I was relatively unusual among educated Americans, because I homeschooled the first two years of high school, then got an associates degree at community college, and was admitted to the state university based on that. Then I went to a graduate program based on undergraduate grades and essays, so I never took the SAT.

Expand full comment

Could you direct me to this post?

Expand full comment

Prediction (80%|somebody runs the numbers): number of surveys completed will be higher in people who reported being depressed.

Expand full comment

Half way through "Attentional Focus and Performance Orientation" it puts "I agree a lot" on the left, with the lower numbers, and "I DISagree a lot" on the right with the higher numbers. This is opposite to basically all the other surveys and most surveys I've taken. It was jarring when I noticed, had already answered one question without noticing. I expect many people will answer those questions incorrectly. I suggest switching it around ASAP.

Expand full comment

I just went back and continued answering questions wrong. It's very unnatural. I think I'd take a 2:1 bet that 20% (or more) of people answer the questions directly opposite there.

Expand full comment

Given that I'm commenting on survey design, I am amused that one of the surveys (the metaphysics one) doesn't offer 1-7, but instead 0-6. That's also uncommon and throws me a bit.

Expand full comment

Yeah I noticed this as well. I got through like 4 questions assuming 5 was agree before I noticed. It was bizarre that they switched halfway through!

Expand full comment

"Scandinavian countries: heavily-regulated market economy" - this seems like a gross over-simplification, one that shows up in annoying political debates and makes them more annoying.

Expand full comment

Yes, I doubt whether Sweden is more "heavily regulated" than for example California by any reasonable metric. Maybe that was true in 1970 or even 1980 but Sweden has spent the last three decades privatising virtually all formerly publicly owned enterprises and massively stream-lined processes and protocols for e.g. starting your own business. It's a modern, middle of the road neo-liberal European welfare state with a public sector that generates a slightly higher percentage of GDP than average.

Expand full comment

Not to be confused with the fact that the mainstream culture is eerily close to a modern American liberal arts college, but that's another matter entirely

Expand full comment
founding

I am frequently amused by the fact that lots of people don't seem to have realized that Sweden, and most of the other Nordic/Scandinavian countries (?), are less "heavily regulated" than they were apparently when they last archived a 'snapshot' of their politics.

Expand full comment

Yes, while I do realise that the Nordic countries don't really make much of a splash in world affairs one would think that at least some people should have noticed that it's not 1950 anymore, even in Sweden!

The other trope that's often still bandied about is "ethnically homogeneous" which is still true of Finland to a certain degree even though that's been changing over the last decade as well but 26% of the population of Sweden consists of immigrants or children of two foreign born parents. About 2/3 of those arriving in the last decade come from outside of the EU as well making Sweden one of the most ethnically diverse countries in Europe including some of the former colonial powers aside from Britain and France.

Expand full comment

Optical illusion survey worked for me and my eyes hurt and I want to slap whoever came up with it. It went on *forever* and it didn't even tell me how many more I'd have to do before I could stop without it chucking my data!

Expand full comment

Sorry 😬

I guess the slapping should bei directed at me (or maybe the researchers from the netherlands who originally invented the stimulus...). If it helps: in the last study that we ran people did like 2 to 3 times the trials and additionally had to listen to a really stressfull acoustic noise stimulus at the same time.

Expand full comment

Thanks a lot though for doing the whole thing!

Expand full comment

I'll accept your thanks if it's offered but I too feel like giving you a bit of a slap. I did reach the end but felt somewhat abused, as if the test was to see how long people who feel as worthless as I do continue a mind-sapping task. Needless to say I finished it, but I felt even worse at the end of it.

Expand full comment

I couldn't even finish it. Felt like it had gone on for way too long to be worth my time. The only thing that kept me going was sunk cost, but even that has its limits...

Expand full comment

I stopped because I thought it was bugged and looping...

Expand full comment

I started to suspect the test was endless and that "optical illusion" was just a misdirection, and really the survey was trying to connect the demographic info in the beginning with something like blind conscientiousness - how long would a person take an annoying test with no end in sight, simply because they were told to?

So I came here to read about it, and I see that's not the case. So, better go finish up I guess 🙄

Expand full comment
founding

I had the same thought, or perhaps that it was stuck in an infinite loop; either way I closed it before thinking to check people's comments here so I will not be finishing it.

Expand full comment

Agreed, I quit after a few minutes.

Expand full comment

The incest survey is hard to do because it's hard to let go of doubts regarding consent and power dynamics and assume everyone is a fully independent and consenting adult as the survey asks you to do.

Expand full comment

It's especially true in the second half which asks about laws, where I can't assume consent can be perfectly discerned, laws have to take into account such practical limitations.

Expand full comment

Like the Theodore Sturgeon short story: "If All Men Were Brothers, Would You Let One Marry Your Sister?" where he examines and argues for consensual incest in an SF setting https://en.wikipedia.org/wiki/If_All_Men_Were_Brothers,_Would_You_Let_One_Marry_Your_Sister%3F

https://www.everything2.com/index.pl?node_id=15273344

I read it years and years ago, thought it was a good story, remained unconvinced by the argument. So maybe I'm stuck on Stone Age morality, but at least it helped me be consistent about it.

"The premise of the story, it is eventually revealed, is that there is a sort of forgotten paradise of a planet, Vexvelt, which is shunned by the rest of the galaxy despite its rich resources and ideal climate. The protagonist, Charli Bux, learns of Vexvelt's existence without knowing why it is shunned. He reaches the planet, is taken into the household of a Vexveltian man, and falls in love with one of the man's daughters. Charli is revolted and outraged to later find her having sex with his host, her father. The father then sits down with Charli to engage him in a Heinleinesque Platonic dialogue:

"Tell me, then: what's wrong with incest?"

"You breed too close, you get faulty offspring. Idiots and dead babies without heads and all that."

"I knew it!...Isn't is wonderful? From the rocky depths of a Stone Age culture... all the way out to the computer technocracies...- you ask that question and you get that answer. It's something everybody just knows. You don't have to look at the evidence....

"Sex is a pretty popular topic on most worlds. Almost every aspect of it that is ever mentioned has almost nothing to do with procreation.... But mention incest, and the response always deals with offspring. Always! To consider and discuss a pleasure or love relationship between blood relatives, you've apparently got to make some sort of special mental effort that nobody, anywhere, seems able to do easily- some not at all."

"I admit I never made it. But then- what is wrong with incest, with or without pregnancy?"

"Aside from moral considerations, you mean. The moral consideration is that it's a horrifying thought, and it's a horrifying thought because it always has been. Biologically speaking, I'd say there's nothing wrong with it. Nothing. I'd go even further..."

Expand full comment

I read an anecdote in which an anthropologist asked an actual Stone Age man why it would be wrong to marry his sister. The answer was that if he married his sister he wouldn't have any in-laws to visit.

Expand full comment

I suspect that even on ACX the data from Question 1 will include a lot of nonresponsive answers and thus be essentially useless.

(To anyone reading this who hasn't seen the survey: Q1 on the incest survey is a very long list of "how concerned would you be about X relationship boinking, provided you knew that it was fully consensual, started as adults, and there was no chance of children?", with no ability to register objections to the premises except general comments at the end. It is blatantly obvious that this scenario is specifically designed to remove all possible objections to incest aside from the divine-command-theory/natural-law "incest is inherently bad". However, I suspect a lot of people who do *not* believe in natural law or divine command theory are still going to answer (or have answered) something other than a long string of "not concerned at all", out of spite at the survey-maker's leading question and/or to screw the data out of something that could be misdescribed as "ACX readers think incest is totally fine".)

Expand full comment

"It is blatantly obvious that this scenario is specifically designed to remove all possible objections to incest aside from the divine-command-theory/natural-law "incest is inherently bad"."

I don't know about that so much, but yes, it's the kind of set-up these thought experiments tend to take when the person proposing them wants very badly to shove you into agreeing with one particular take. Like The Violinist abortion thought experiment, where the article goes even battier with "imagine you get pregnant by something like dandelion fluff flying through the air, now would it really be bad if people put up screens on their windows to stop this happening?" Yeah, and if my backside was made of solid gold, I could just carve off a chunk now and again when I needed money. But it's not.

The incest one is trying to get at "is there an underlying reason why we think it's bad?" by removing all the potential child-abuse and rape elements, but that's misleading since nearly everyone agrees that abuse and rape, regardless of whether it's incest or not, are bad.

Expand full comment

>I don't know about that so much, but yes,

Are you referring to the argument about joining families? I did miss that one, although the premises and other conclusions of that argument mean that it's only especially relevant to a neoreactionary (modernity violates the "stable extended family as primary unit of deep social interaction" and "chastity outside of marriage" premises and tends to reject the "celibacy is bad" and "in-laws are also incest" conclusions; religious conservatives don't need this argument because "don't engage in incest" is baked into most religions as a divine commandment and thus the DCT objection applies).

Or are you referring to the strength of my language about "blatantly obvious" and "specifically designed"?

Or something else?

Expand full comment

Okay, I got through most of them, skipped a few that weren't aimed at me or were very difficult to understand. I will accept your praise slash concern.

Expand full comment

The personality survey bundles together things that I feel shouldn't be bundled together at all. For example, I do feel "down to earth" but not unimaginative. I'm ethical, but not unforgiving. I answered natural on these question though cause if they were separated i would give opposite answers.

Expand full comment

Yeah, some of them were "well I do feel a but not its companion b", so I went for "which of this dipytch most resonates with me?" (if you'll excuse the pretentiousness).

Expand full comment

Does the Optical Illusion one record my results if I stop partway? I eventually gave up and quit. I'm not even convinced that it was ever going to stop.

Expand full comment

Yes, ist does! Thanks a lot for holding up as long as you did :)

Expand full comment

Ok, that makes me feel a little better about doing it, but stopping when it got unbearable.

Expand full comment

I'm surprised you didn't add Georgist to the list of Political Affiliations after the book review contest winner.

Expand full comment

I think it can discourage many people that the survey is hosted by Google Forms. Even though you do not type in a name, Google has numerous way of tying the answers to your identity and will have them stored for god-knows-how-long. Given the personal nature of some questions, it seems to have potential for abuse. Given how trivial it is to self-host a simple survey form, it would be the respectful thing to do w.r.t. your users.

Expand full comment

Not wrong but I would propose that Google probably has a rather good (better) idea from your search history, anyhow. Unless of course you are super diligent at anonymizing that.

Expand full comment

Yes, that seems plausible. I use startpage.com for search myself but I understand that for a majority of people this may be true. Another treasure trove of personal data might be gmail. But inferred data may have less weight than explicit and detailed answers. For example if you explicitly answer some questions about sexual orientation, political ideology etc. and some repressive government later gets hand on the data it may (or may not) be worse than simply searching for related terms. Not sure.

Expand full comment

I had to give up on #2. I have no brothers.

Expand full comment

21 is a very unpleasant experience and takes too long - the visual illusions made my head dizzy and my vision blurry. I tried keeping at it for as long as I could, but I just couldn't finish it.

Expand full comment

I've got it paused in an open tab overnight, and am starting to wonder if the real experiment is about the extent to which depression symptoms effect how long users stick with it.

Expand full comment

You should change "Alt-right" to "Nationalism" next time around, what you're describing as alt-right is simply nationalism.

Expand full comment

Oho, I'll fight you on that one! I'd happily describe myself as a Nationalist, but that does not mean I'm Alt-Right:

https://en.wikipedia.org/wiki/Irish_nationalism

I very much dislike the attempts by interested parties on both left and right to try and bunch in together Patriotism, Nationalism, and the Alt-Right/Neo-Nazi, in order to respond to cases of "you want to fly your flag outside your house?" so that instead of going "okay, bit enthusiastic when it's not for a football tournament but you do you", they go "you are clearly a member of the KKK who celebrates Hitler's birthday!"

Expand full comment

Which leftist was telling you that flying the Irish flag means "you are clearly a member of the KKK who celebrates Hitler's birthday!"? Most communists are above that, thankfully. You should look into Marxism-Leninism as a good leftist outlet for your Irish nationalism. Ho Chi Minh was a great nationalist and communist, for example. Many great Irish communists too!

Expand full comment

I'm not saying that nationalism=alt-right, only that the description he provides of "alt-right" is simply a description of nationalism.

Expand full comment

Do you strictly want this to be a reader-only survey? What if this blogpost is someone's first ACX read - would you consider them as being an 'ACX Reader'? I want to let readers of my newsletter know about this survey but I do not want to adulterate your survey with non-readers. Let me know.

Expand full comment

One of the questions in survey #0 (general demographics) asks how long the survey-taker has been reading ACX. Anyone you referred to this post would presumably put down "less than a month", making it easy to filter them out (or in!) in cases where it matters whether they're a long-term ACX reader. So I think you can just go ahead.

Expand full comment

I did see that and you make total sense. Thank you!

Expand full comment

I don't know what to enter in the "work status" question. I work for a UK University. Like almost all UK universities, it is an independent but largely publically funded institution. It is technically a charity. So it's not for-profit, nor technically government, but it's not normally perceived as non-profit.

In the UK, we'd normally describe universities as "public sector", so I'm guessing that translates to "government" in USA-speak, even though we're not actually part of the government.

Expand full comment

Also, should survey 2 be flagged in the summary list as targeted at homosexual men?

Expand full comment

I wrote this one. The targeting would be OK, but I want to compare demographics of families with and without homosexual men, so I don't want to exclude anyone.

Expand full comment

Oh good. I was concerned that I'd submitted a response when I wasn't the target audience.

Expand full comment

Regarding the ACX survey (24), going through it clarified my thoughts about the newer blog and Scott. I would definitely pay for a published compilation of the old essays & fiction, but I don’t read this blog with the intensity I read SSC.

Nowadays it’s a casual read. I like to read professional writing that’s not in my field, and I read In the Pipeline for a similar reason. SSC essays had a style wit and presentation that I really liked.

Expand full comment

General demographics "profession" subtype computers: need to choose either "practical: IT, programming" or "AI". I object to the claim that AI is either not "practical" or does not require "programming" - I do practical programming of ML systems, which is what almost everyone means by AI. Does the survey mean "AGI hard research"?

Expand full comment
founding

I'm just guessing, but I imagine that was more of an attempt at a sub-type within the constraining UI of a single flat set of options than any kind of claim about AI work being neither practical nor involving programming (or 'IT' work).

Expand full comment

In survey #4, "Digital Literacy", it asks you whether you're familiar with a list of technical terms. One of these is "proxypod", which is presumably a fake term thrown in to check if I'm just clicking "yes" to everything - but it sounds to me like it *should* be a brand name for a low-power VPN server.

Expand full comment

There's a podcast called Proxypod, fwiw.

Expand full comment

Yeah it sounded like a brand name for something like a Pi-hole with more filtering.

Expand full comment

In survey #0, "General Demographic Information", it asks how many children you have. For men, in particular, it's possible to be uncertain of this, but the field requires a specific number rather than a range.

Expand full comment

Ah, so what you are asking for as regards men and children is the "Boris Johnson option"?

See Wikipedia entry on our latest "who knew he was Catholic?" co-religionist:

"Children At least 6[a] Johnson has not disclosed how many children he has. He is known to have had four children with his second wife, Marina Wheeler; a fifth child from an extramarital affair with Helen MacIntyre; and a sixth child with his third wife, Carrie Symonds."

There was also one affair which resulted in an abortion and a miscarriage, so that is two children more if they had survived. Plus other affairs but we don't know if pregnancies ever resulted.

Expand full comment

That strikes me as an odd reply. In order to shoehorn some political issue into the conversation, you've had to ignore the point of my comment: I'm talking about the scenario in which the number of children is unknown to the (possible) parent themselves, not to third parties.

Expand full comment

It was meant to be a joke, rather than political; the current most-high profile (that I heard of) guy who's not sure exactly how many kids he has because he's had the chance to have a lot of them.

Expand full comment

Hey, this is kind of a weird comment, and I don't want this to sound preachy or anything but I am a little disappointed in the categories when it comes to the profession tag.

I currently work at an Amazon warehouse as an associate. Not one of the full-blown fulfillment centers, but one of the smaller delivery stations. While I have a pretty firm way of how I would describe my job to others: "Warehouse Worker", there is no option like that here. Okay so I should check "other blue collar." Fair enough.

But what if I still had my old job as a Dollar Tree cashier? What would I have put then? It is neither white collar nor blue collar under traditional definitions, yet not only do a majority of the people I know work in the retail industry, according to the Bureau of Labor Statistics, it is still one largest industries in the country in terms of people it employs. Keep in mind that Walmart, Amazon, and Kroger are the largest private employers in America last time I checked, and it's no coincidence that all of them have large retail/grocery components.

And I understand that a majority of readers who read this blog are more academically/IT inclined and you would want to capture the diversity on that front as much as possible, but I think you lose something when you have three different categories for IT and only two broader "Other" categories for what a majority of working Americans actually do. There could be interesting differences between the retail workers and warehouse workers that now won't show up because of the broadness / ambiguity of the definition. I know I have a lot more back pain working at Amazon now than I ever did at Dollar Tree. You can't even write in an answer to clarify the "other" so it is a difference you may never know.

Again, sorry for the rambling, I've been a fan of this blog and its predecessor, but that element of survey has always rubbed me the wrong way. I understand that it may be too late to substantially alter it now, or there might be space limitations, or other good reasons to keep it the way it is that I haven't thought about. Whatever the case is, I just thought I would mention it.

Thank you for your time.

Expand full comment

Working on the till is more "pink collar" work, as it has traditionally been women working there, and having done that line of work myself I'd lump it in with "other blue collar" for want of a better category.

The jobs category on these surveys always makes me smile because it is so plainly heavily influenced by "Bay Area friends and associates" who have seven distinct categories to go with "computers" but seemingly never did public-facing work outside of an office, if even there. Good luck to them! There's a reason "customer service smile" is also "secretly thinking murder smile" 😀

Expand full comment

I did a six month stint as a cashier in a department store. I understand your secretly thinking murder comment.

It was stressful to maintain a pleasant demeanor in the face of some really bad behavior but it was a great educational opportunity too.

I got to see a wide swath of humanity that I would never had contact with.

Everyone from the saintly to the just plain mean passed in front of me.

People with severe disabilities maintaining a presence of serene dignity.

Small acts of grace. Someone using their own cash to cover a purchase that a mentally disabled person was trying to make with a cardboard credit card.

Men exposing themselves to female employees.

A poignant written conversation with a charming mute woman.

Cross dressers claiming they are just doing it for a joke.

Professional shoplifters known to the security team that somehow always evaded capture

And one grand theft a $10,000 dollar stone.

Don’t ask me what a ring of that value was doing in a Midwest Sears jewelry department.

Expand full comment

My stint at the coal face definitely helped me be more patient and understanding of people working those jobs - the cashier or the waiter or the receptionist or the customer service person on the other end of the phone *doesn't* have autonomy to make decisions about taking cheques, coupons, special offers, or the likes. Yelling at them does not help.

It also let me see a side of people who, as customers, really do treat you as less than the mat they wipe their shoes on.

Expand full comment

Is coal face an idiom?

I came away from my experience thinking it would be a good idea for everyone to spend some time at that sort of job. Might increase the net sum of empathy in the world.

Expand full comment

Ah, yes, sorry, it is! I use British English so yeah:

"at the coalface

UK

doing the work involved in a job, in real working conditions, rather than planning or talking about it:

At the coalface with a deadline looming, you sometimes feel under a lot of pressure."

Expand full comment

Got it now. Thanks.

Expand full comment

I bet you study maths and read the sport page too. :)

Expand full comment

Okay Googled coal face idiom. I was taking it literally. I was never a colier but did do a decade in an iron mine.

Expand full comment

Huh, I had never considered the "pink collar" aspect. In most of the store I've worked, we had a pretty balanced gender ratio, but I suppose in the past it would have been different.

In the modern context, I think the phrase that is most accurate to describe these jobs is No Collar. I have yet to work at a workplace that had a dress code , even at Amazon. They just want you to wear the safety vest over whatever else your are wearing and that it wasn't too baggy to get caught in the conveyor belt. So I usually just end up wearing generic T-shirts.

Then again, most from what I hear most IT places are much more relaxed about attire now as well. Maybe we are all No Collar now.

As for the discussion about the quality of customer, I have actually had a pretty good experience. I think that comes with working at a Dollar Store. Everyone kind of knows what to expect, so there isn't much opportunity to get upset. Although, I did have to price check items a few times too many in store in which literally everything costs a dollar.

Expand full comment

I answered three surveys: one assigned, one out of interest, and one about Covid that turned out to be pointless because I live in New Zealand.

Expand full comment

All surveys complete! Except the optical illusion one which wanted a cookie; I see others have had the same problem.

How to make your ethnicity surveys Brit-friendly, by a Brit:

* "Asian". In North America that means East Asian unless you write in South Asian. In Britain it means South Asian unless you write in East Asian. So a South Asian Brit who clicks Asian will find themself categorised with the American East Asians instead of the write-in South Asians. I know you don't want 10 options plus a write-in box, but I'd consider splitting those out.

* "Hispanic". In Britain, Hispanic is a fancy word for a (typically white) person from España - c.f. Gallic, Teutonic. "Non-Iberian Hispanic" is a useful modifier, or just call it Latino (which also avoids the arguments about whether Brazilians are Hispanic).

* "White/European". Try to avoid this one. Millions of people of colour live in Europe and are proud Europeans, and with Brexit and the racial abuse of England's non-white footballers at the Euro Championships being hot topics, it's a bad time to equate not-White with not-European. "White/Caucasian" is better; "White" is perfectly good.

Unrelated to race: the average Brit doesn't know what an FDA is. I would hyperlink FDA and put "or your country's equivalent".

The average Brit also doesn't have interesting opinions on American politicians or the state of American discourse. Can you let us know whether to answer for Britain or to skip those questions?

Expand full comment

Brazilians are Portuguese which are Iberian not simply Hispanic, though by American categories that probably does count as Hispanic or at least Latino.

Now I'm wondering about Goan Indians - that's perfectly possible to be Portuguese and South Indian, see the Kipling story https://en.wikipedia.org/wiki/Kidnapped_(short_story)

"The manner of his fall was in this way. He met a Miss Castries - d’Castries it was originally, but the family dropped the d’ for administrative reasons - and he fell in love with her even more energetically than he worked. Understand clearly that there was not a breath of a word to be said against Miss Castries- not a shadow of a breath. She was good and very lovely - possessed what innocent people at Home call a ‘Spanish’ complexion, with thick blue-black hair growing low down on the forehead, into a ‘widow’s peak,’ and big violet eyes under eyebrows as black and as straight as the borders of a “Gazette Extraordinary,” when a big man dies. But- but- but- Well, she was a very sweet girl and very pious, but for many reasons she was ‘impossible.’ Quite so. All good Mammas know what ‘impossible’ means. It was obviously absurd that Peythroppe should marry her. The little opal-tinted onyx at the base of her finger-nails said this as plainly as print."

Expand full comment

Uh, there are an *awful* lot of Brazilians who speak Brazilian (mutually intelligible with Portuguese) but aren't descended wholly or mostly from Iberians. Iberians in Brazil are the old money, the social elite. If you want Brazilian to be a subset of something, it's Latin American rather than Portuguese. It's pretty much the same distinction I was making between Hispanic and Spanish Spanish.

Expand full comment

Yeah, I understand that. But if we're going to use Continental European names to describe South American racial categories, then Portugal is not Spain, Portuguese are not Spanish are not Hispanic, and the Iberian Peninsula is where it's at.

Take the actor Ricardo Montalbán, who actually was Hispanic by virtue of being born in Mexico to Spanish immigrant parents, but by American catergories was not "Hispanic" as that meant a different sort of Mexican heritage (he'd be "White Hispanic" under current census labels).

Now when getting into the weeds of "chocolate/coffee/cream" model of the Great Brazilian Social Layer Cake, those categories are little to no use. So Latin American, poor as it is, is the most useful term.

Expand full comment

So that's exactly what I said, with the footnote that Iberian *isn't* particularly where it's at with Brazil. There have been more Italian migrants to Brazil over the centuries than Portuguese.

Expand full comment

Wouldn't the Asian category include both East Asians and South Asians, regardless of which comes to mind first?

I haven't taken the survey yet. If it has, say, "Asian" and "South Asian" as separate categories, I agree that's confusing. But if the only Asian category is "Asian", I don't see the problem.

Expand full comment

Indeed, but the fact that Asian is a single category is because of demographic assumptions that only make sense in North America - that there are far more East Asians than South Asians, and that the South Asian experience of life in America is close enough to the East Asian experience (math prodigy whose parents owned a restaurant?) that it doesn't matter. Whereas if Brits and Americans had to sit down and agree on what races exist, they'd soon conclude that they were talking about two different races when they said "Asian". There are a good 1.5 billion people on each side of the South Asian/East Asian divide, and they are as different from each other in appearance and culture as black Americans are from white Americans.

(Hans Rosling has a brilliant graphic in his book: extremely roughly, there are about 1 billion people in the entire Americas, 1 billion in Europe, 1 billion in Africa, and 4 billion in Asia.)

Expand full comment

Thank you for this!

Expand full comment

One of the surveys had Agree on the left and Disagree on the right such that "1" was strongly agree and "9" was strongly disagree. I wonder how many people are going to have read the words above the numbers?

Also the metaphysics survey was way to hard for 1:30am

Expand full comment

JohnWittle, your astralcodexten user ID is s7wvK3jg9XFHpqqX

just google 'site:astralcodexten.substack.com johnwittle survey user id' to find this comment next time you need it

Expand full comment

"Answers will be aggregated to perform anonymous statistical analysis. Your individual answers will also be made public so that others can view them and perform their own analyses. When answering questions, please do not submit information that you do not feel comfortable making public."

Oops! Wish this warning weren't JUST on the first survey, and, you know, wish you hadn't told us to take it out of order.

Expand full comment

The "Family Demographics And Homosexuality" survey is confusing because it only asks for ASAB, not gender, and then it asks "Are you exclusively homosexual?". I'm not sure what is meant by "homosexual" in such a context.

Expand full comment

I considered phrasing it as "Are you an obligate homosexual?" but thought that might confuse people more. For all intents and purposes, "Do you strictly prefer sexual relationships with people of your own sex?" would be OK.

Expand full comment

The problem is it's not clear what you mean by "your own sex". In the LGBT community, "homosexual" usually means "attracted to people of the same *gender*". But your survey doesn't even ask about gender. So I'm *guessing* you mean chromosomal sex, but I'm not sure.

Expand full comment

That is, I'm guessing you meant "attracted to people whose appearance corresponds to your own chromosomal sex", since obviously attraction depends on appearance not chromosomes.

Expand full comment

Right. Are half-brothers to be included? Same treatment for 'via father' and 'via mother'?

Expand full comment

This was a significant oversight on my part, I'm realizing. The theory around the questions suggests inclusion of maternal half brothers and full brothers and exclusion of stepbrothers and paternal half brothers.

Expand full comment

#13 is for psychedelics users only. Is #22 for everbody?

Expand full comment

So I did a bunch of these yesterday. Early this morning I have this long evolving dream. I’ve taken a new job and my new employer is administering these these endless seemingly pointless multiple choice tests to see where I’ll best fit in with their teams.

Oy. This is never going to replace my favorite. I’m the captain of a pirate ship, straddling a canon. My crew is the University of Minnesota women’s volleyball team. I can’t help but think that one means something…

Expand full comment

Does the cannon shoot volleyballs?

Expand full comment

Could be. Not really important as long as the crew has long well toned legs.

My wife hates when I make this joke. :)

Expand full comment

And to practically invite some sort of censure

https://mobile.twitter.com/GopherVBall/status/765615820676272128

Expand full comment

Appalling! I censure you, sir!

Expand full comment

From 'Much less than normal' to 'Much more than normal' -- does that ask the personal 'normal' (healthy, i.e., 'Me before X'), personal average, or a population average?

Expand full comment

Incest survey doesn't ask for user ID; intentional?

Expand full comment

The user ego and superego are more needed in that situation.

Expand full comment

I had one of those awkward both/neither reactions to the incest survey.

I want a blanket prohibition on sibling incest. I don't want the legal code to have to get into the minutiae of which sex acts are illegal under which conditions; that's just an invitation to lawyers to find loopholes. I also think sibling incest is a *bad* choice, and don't want people advocating it any more than I want them advocating smoking.

But under the exact conditions specified in the survey, I would issue guidance to the DA/CPS that it isn't in the public interest to prosecute anyone. So the "restrictions" wouldn't actually restrict anything.

Expand full comment

I'm in much the same boat; if I want a ban, I don't care if it's "they can't get married but apart from that they can fuck away to their heart's content". On the other hand, there are things I disapprove of, but don't see that they can be realistically banned, since they already are or previously have been legal (e.g. uncles marrying nieces). Or the vexed question of the "marriage with deceased wife's sister" which was finally permitted in the UK in 1907 https://en.wikipedia.org/wiki/Deceased_Wife%27s_Sister%27s_Marriage_Act_1907

but which seems to have already been in practice, given the amount of "after his wife died, Jeremiah Jenkins married her sister Ermentrude who had come to look after the household and young children" I see in accounts of various Victorians. Which makes it entirely possible for households to have "my half-sibling is also my first cousin" families.

Expand full comment

I encountered that in Gilbert & Sullivan and never saw why it was controversial. You're clearly not allowed to marry a half-sibling even if they're also a cousin.

Expand full comment

Yes, the "perennial blister" in G&S. The controversy was "can a widow/widower marry the sibling of their deceased spouse?".

The controversy was because the Church of England was maintaining the old degrees of consanguinity, under which such marriage was incestuous (because man and wife are one flesh, hence their siblings are now the same as your siblings). This was carried over from mediaeval church practice. The problem was:

(1) On the one hand, this had been part of Henry VIII's case for annulment of his marriage to Catherine of Aragon - she is my brother's widow, this is incest, this is why God is punishing me that I have no living male heirs. So the CoE couldn't very well junk it.

(2) On the other hand, over the course of centuries and the way the CoE had been pulled about during Reformation period, with theologies swinging from "Catholic but not Roman" to "Low Church", plus the continuing rise of Dissenter/Non-Conformist Protestantism, and the way that marriage had been functionally reduced from a sacrament to a human tradition, plus you know, permitting divorce (even if in very limited circumstances)

And

(3) Like it or lump it, there were plenty of people already wanting to marry their sisters-in-law/brothers-in-law

Then the question was always being raised and always being turned down, until eventually in 1907 they formally gave in on it and say "why the hell not? it's not really incest".

Expand full comment

The constant raising and turning down would make me think it had become a proxy for how religious the law ought to be in general.

Expand full comment

I imagine so, society having moved on and all that. The CoE being a state church with bishops having seats in the Upper House of Parliament probably meant it felt it needed to set an example and maintain standards, even if the population was happy to scrap those standards.

Expand full comment

It would have been nice to be able to give direct feedback about a survey to the survey author. I realize I can comment on it here, but don't know if that will get through. For example, after answering several questions on the "Health Related Quality of Life" I felt the comparisons weren't specific enough for me to give accurate answers (so I decided to bail out rather than skewing things)

Expand full comment

In 21 (optical illusion), I see the red cross and then it all goes black, which seems to me like unintender behaviour. I'm using Brave browser. I have already answered the first two pages of questions, so it's annoying to test with another browser.

Expand full comment

I said I was "hispanic" given the options, but that didn't feel like a "race" question to me, but nationality. The alternative being "white (not hispianic) [where hispianic is how americans call people from certain places]".

Expand full comment

Why not set up a site to do the randomization and redirects automatically?

Expand full comment

I found the 'Personality' survey very frustrating to take. Most of the questions seemed ambiguous at best. Do they want us to answer based on BOTH words, or based on EITHER word? For instance, 'active' and 'talkative'. Do they want a response to 'active AND talkative' or a response to 'active OR talkative'? Most all of the questions suffer from this defect. What if I'm talkative but not active? What if I'm carefree but not amoral? The author apparently feels that words like 'emotional' and 'anxious' are synonyms or something. Not true.

Expand full comment

It prolly wants to force you to think about where the intersection of the two words would be, so you aren't just reacting to the positive/negative connotations of a single word.

Expand full comment

It seemed deliberate, though I know not to what end.

Expand full comment

I quit 17 minutes into the Optical Illusion test that supposedly takes 10-15 minutes. I hope it at least saves the data part way.

That test *seriously* needs to be a) much shorter and b) have a progress bar. After a while, I started to suspect that it was secretly a test to see how long keep would keep doing the same thing over and over before giving up.

Expand full comment

I suspected the same thing, and felt progressively worse about myself for not being assertive enough to stop.

Expand full comment

I guess you were very close to the end then and we can still use the data so thank you very much!

And, no, it was not a patience test :D

We actually already reduced the trials to the minimal amount with which you can still analyse the data in a meaningful way...

Expand full comment

"Understanding your beliefs" had several US-specific questions that took me fairly flat-footed. I live in Canada.

Expand full comment

Same for "Moral Curiosity" ending with impressions about "Democrats" and "Republicans". It is interesting for the author of a "Moral Curiosity" survey to intuitively feel that US political parties are a dominant category of moral concern in a worldwide survey.

Expand full comment

If you live in the English-speaking world, you've heard the media chirping at you about January 6th and how the GOP are now the greatest threat to democracy and yada yada. That counts as an impression. There are Americans whose impression extends about that far.

Expand full comment

I just took all applicable surveys except for the optical illusions one, at a relaxed pace. It took me somewhere around 3 hours total, on a lazy Saturday afternoon.

Expand full comment

I completed every survey except that fucking optical illusion survey. It said 10-15 minutes but it felt like an eternity.

Expand full comment

I gave up on it after 17 minutes. It's ridiculous.

Expand full comment

Perhaps it's actually testing patience and time preference? It could then try to correlate how long you waited with nationality, age or profession.

Expand full comment

I considered that but I can't imagine that would pass an ethics review

Expand full comment

Same.

Expand full comment

I am probably skipping the survey this time because of the trivial inconveniences that exceed my threshold. Instead of filling one long form, it's "create a weird id, copy paste it, keep it some place for every other form, insert it into the next form..."

I wish it was more like "this cookie will be used as your default id for the survey, you can edit/replace/delete it as you see fit, should you care enough", then each form would end with a link to the next one, id-preserving.

Expand full comment

The ACX Customer Satisfaction Survey made me think about why I considered becoming a paid subscriber to ACX, but chose not to (so far). I certainly enjoy ACX, am a paid subscriber of other things, and could easily choose to subscribe to ACX.

After thinking about it, I'd say the main reason is actually Scott's repeated downplaying of the importance of subscribing, and of its benefits. It's basically always presented as *super* optional, with an underlying message of "don't feel bad if you can't subscribe, it's really not that important, all of the good articles are free anyway".

I understand why it's presented that way, to avoid some people's scrupulosity issues, but it's honestly so much of a downsell that it makes paid-subscribing sound like an unattractive option. It sounds like "if you *really* want to throw away your money, then be my guest, I guess".

Expand full comment

I would have like an option that said "It's too expensive for me" rather than saying it's too expensive in absolute terms.

Expand full comment

I expect he's trying to avoid the implication that you pay $10 and you get $10 worth of stuff, and if you haven't gotten $10 worth of stuff this month then you should get mad at him.

What you get for $10 is that Scott doesn't have to get a job other than writing ACX.

Expand full comment

It seemed like he didn't want people with tight budgets to feel like they had to choose between reading ACX and whatever else they might do with $10/month, which I appreciate. Personally, I subscribe to no media at all outside of two streaming services, for purely financial reasons. I agree with Phil Getz that it would be more accurate to have two options: the price is to high (in general), or it's expensive for me.

Expand full comment

I am sure this was discussed to death already, but I lost the memo. What does Hispanic mean and how is that in the race category? I am Argentinian and so I don't identify as white (mainly because all white countries have decided we are not white, so I follow suite here and do not want to identify as white) but neither do I identify as "hispanic". In the actual hispanic language, that means Spaniard, which I am not. In the unitedstatezen language, as far as I know, hispanic means "Mexican looking", which does not apply to me either, neither by personal identification nor, I believe, by the standards of the natives that came up with that word (I've had 4 Unitedstatetenzens, as well as 7 Europeans and dozens of Chinese, tell me straight to my face: "ohh, so you are Argentinian but also obviously (never understood why is that obvious) not Latino/ not Hispanic, what race are you?"

I chose White, at the end. Does that mean that all my survey answers will be read as belonging to some anglosaxon protestan puritan or similar? Probably not very important, but in my mind that would disqualify the results at least a little bit.

(I really would like to know what Latino/Hispanic is suppossed to mean and why I so obviously do not belong in that to the people that came up with those words. Actual Latinos/Hispanics do not use those words nor do we feel we are a "race")

Expand full comment

"Hispanic", in the Unitedstatezen language, means people from Spanish-speaking countries in the Americas, and their descendants. We're aware that this doesn't really make sense as a racial category, but we still think of it that way.

In the U.S. census, I'm "non-Hispanic white", and you would be Hispanic.

Expand full comment

"Hispanic" refers to language/culture/ancestry/something. Hispanics can be white/black/indigenous. It's possible that including Hispanic along with what are otherwise racial categories is evidence that the whole thing is silly, or at least culturally contextual with no reference to anything other than what people tend to imagine.

Expand full comment

Most "hispanic" people in the US have significant indigenous ancestry, so it's not quite as crazy as it sounds.

Expand full comment

It's not as completely crazy as it sounds, but it's much more about culture than considering white or black to be races.

Expand full comment

I love all the comments from people not in the U.S. about the demographic questions because it does so underscore how nuts and oddly specific U.S. census-type race and ethnic categories are.

Is someone from Portugal (who is now a U.S. citizen) hispanic?

Expand full comment

No. Not even someone from Spain is Hispanic, even though it's right there in the name.

Expand full comment

Is there any hope for these categories? What if instead we asked people to pick the top three "of X descent" boxes that they feel apply to them?

Expand full comment

I don't understand why people making surveys about US politics and race don't mark them as US only. What are they thinking? (Probably nothing)

Expand full comment

Just here to boast - did all of the ones I was demographically qualified for... except the optical illusions one, I couldn't get that one to work (chromium, Linux).

Expand full comment

Nice! Thank you!

Expand full comment

I've been reading OB/LW from way before LW even became a thing. I feel like a large part of the impact that cryonics has made on rationalists stems from its discussion there. My life doesn't revolve around cryonics, but I'm sure I've read most, if not all, of the posts about cryonics on OB and LW. **I can't recall ever hearing the term "biostasis"!** Using Google's site search, there doesn't seem to be any mentions of the term on LessWrong until this year...

Expand full comment
founding

I too started reading back in the Overcoming Bias days yet wasn't familiar with this usage of "biostasis" so I googled it before answering. The wikipedia article didn't mention cryonics, but from biostasis.com it looks like "cryonics" is trying to rebrand itself under "biostasis"? Probably not a bad idea, if so.

(Also, the knowledge questions were hard! All I knew was that cryoprotectant prevented formation of ice crystals that would otherwise scramble the cells -- I recall seeing micrographs of frozen vs. vitrified tissue -- but there wasn't even a question about this, unless it was an implication of the final question about thermodynamics.)

Expand full comment

Yeah, I also had trouble with the knowledge questions.

Expand full comment

Someone tell Scott that there needs to be a "got pregnant and then gave birth" option on the weight gain/loss survey. Pregnancy and childbirth are the only reason I've ever gained or lost weight, but my weight history looks completely bizarre without noting "gave birth then."

I don't understand why pregnancy wasn't mentioned/asked about in the survey since it is probably the single biggest reason why people suddenly gain and then lose more than 20 lbs in a short amount of time.

Expand full comment

If that makes you feel better, COVID wasn't mentioned too and I've heard a lot about "lockdown pounds" or "pandemic pounds", so you're not alone in being forgotten about.

Expand full comment

Came here to say the same. Rather myopic to miss this.

Expand full comment

Survery #4 is rendering oddly. "of trying to gureit out myself." it has an up arrow instead of "Figure it"

Expand full comment

Also the Likert scales on #7 (Meditative Experience) are backwards -- filled in quite a few incorrectly before noticing and fixing.

Expand full comment

I completed all of the non-targeted surveys except 7,(which may merit a "targeted at people who have ever meditated note, I quickly grew bored with skimming through long lists of things that read as nonsense to me just to check the "no" button), and 21. (Took way too long and felt like no actual end was in sight, beginning to agree with the suspicions posed by other commentators that the whole thing was an exercise in testing patience.)

Of the set, I thought 16 was the most interesting. I'm looking forward to seeing investigations of those results, and what the intent of the survey was.

Expand full comment

Just ran the table!

Expand full comment

Survey 5 "targeted at people with depression/anxiety" -- The questions all relate to symptoms and treatments for depression. It instructs a person with anxiety to word-substitute anxiety where it says depression throughout the survey.

I don't think doing that is going to yield meaningful data about people with anxiety. Anxiety and depression not uncommonly co-occur, but they are different things with different symptoms, different coping behaviors, and different treatments.

Expand full comment

IIRC the first question asks you which of depression and anxiety you have, and suggests word-substituting anxiety only for people with anxiety and not depression (there's also user ID, which hits the separate questions in survey 0).

As such, the groups should be separable in analysis.

Expand full comment

i'm not sure if it was there and i just missed it, but i don't think the meditation survey asked about *when* you experienced the weird religious-type experiences it's asking about, or the context in which they happened? i'd venture to guess people with, for example, charismatic christian backgrounds or schizotypy would have plenty of experience with weird religious-type experiences.

Expand full comment
founding

The weight loss survey included questions about life events that looked like they were trying to approximate something like the Holmes and Rahe stress scale, but it was missing the most impactful questions (e.g. "death of a spouse", "divorce", or "imprisonment"). An oversight, or did the survey had other things in mind?

Expand full comment

Survey 8: “Almost all politicians are liars”

Is there a difference between lying and deception? The former seems more specific to me.

Expand full comment

I agree that lying is more specific than deception. But they're still almost all liars.

Expand full comment

Nitpicking, but France's National Front doesn't exist anymore (survey #0). It has been replaced by the "National Rally".

Expand full comment

So i gave up on the optical illusion one because it seems to go on forever with very similar or identical optical illusions. Did anyone actually complete it?

Expand full comment

As a data point, I also gave up that one.

Expand full comment

Thanks for completing it/doing it as long as you did (althoug it is so annoying), every data point helps!

Expand full comment

I completed it, but only because I was alt-tabbing to it while waiting for opponents in a video game. I probably would have gotten a lot more tired of it if I had to do it all in one go.

Expand full comment

I completed it in about one sitting (after freezing my computer with it twice because the computer couldn't handle both the survey and an unholy number of tabs — this should not be a problem for most people, but it did mean that I ultimately filled out the questionnaires at the start three times), because I had read some of the comments saying it took a long time and was prepared for that. I was actually surprised when it ended, because I had expected it to go on for longer than it did (yes, it *does* end eventually!).

Expand full comment

I ran into on the first trial of optical illusions after the practice trials. In case this helps debug: "Error in Fixation Cross (eventAPI.js:101): during event after:end : EncodingError: the source image cannot be decoded."

Expand full comment

What with the amount of complaining here, I haven't even tried out the surveys. Are there any surveys people think are well-designed?

Expand full comment

Haters gonna hate. I did like ten surveys and all were fun IMO and was better designed than most surveys I've taken. "23. Metaphysics" was a favorite.

Expand full comment

I am having trouble with the COVID questions. These seems to be no option for "I absolutely did not care at all". I have never been even the slightest bit concerned that I would "get" COVID or be harmed in any way by it if I did. Biologically, it doesn't even exist as far as I am concerned. So questions like

"During the Covid pandemic, which worried you more (choose 1):"

Are meaningless.

Did you model these after the tautological and undecidable "COVID questionarres" that have to be filled out all the time for insurance coverage?

-- "In the last 14 days, have you possibly been in contact with a person who may have been exposed to COVID?"

Uhhh...how can I possibly answer that except with "Of course, that is possible"?

I feel I may have to punt this questionarre because I can't answer accurately.

__________________________________

Let me go further, because I feel like my perspective is unique and might have some value to the community, which is why I hope it can be capured in the survey(s)

[REDACTED FOR BREVITY]

BRetty

Expand full comment

This survey is so much worse than previous surveys that I failed to do more than the first two and gave up halfway through 3. I put this down to the hassle of the user ID, having each part separate without indicate who should fill them in (2 is only aimed at men),the awfulness of the questions in 3 (as a disabled person they made me feel physically sick) and the overall lack of sensitivity (how hard would it be to include a non-binary option for instance). I've lost my user ID now so I won't be doing more but I though you might like to know, as the survey was never this bad before

Expand full comment

FWIW, most of these are not Scott's usual survey, but surveys by readers that he is hosting as a favor to science, or whatever.

> the awfulness of the questions in 3 (as a disabled person they made me feel physically sick)

I'd be curious to hear you expand on this. Would there be a better way to ask about the stuff #3 is trying to get at, or is the whole survey misguided, or what?

Expand full comment

Regarding #19 (Attentional Focus And Performance Orientation):

The second half of the survey gives a 1-5 scale with 1 labeled as "agree a lot" and 5 labeled as "disagree a lot". This is the reverse of what I would normally expect, and I almost didn't notice that higher numbers didn't indicate stronger agreement. I answered according to the provided labels, but other people might not notice the change or might assume it was a mistake and answer the reverse.

Expand full comment

Woops, I should have read this comment section, I wrote pretty much the same thing. :D

Expand full comment

The question about health reminds me of https://xkcd.com/883/ -- it is easy to remember all things that are wrong or suboptimal with my body, but when I start to think about all the problems I *don't* have, or just list all organs in human body I know about and then consider how many of them never gave me a reason to complain, suddenly I feel quite healthy.

Now sure why should my weight be correlated to what I did in 2018, but whatever.

Note to survey authors: if you make a scale with both numeric values and verbal descriptions, please make "less" correspond to smaller numbers and "more" to larger numbers, otherwise it is a bit confusing. ("How much do you like kittens? 1 = a lot, 7 = just a little")

If your survey is long, please split it into pages of similar length and always say "page X of Y". It sucks when I am at page 10 and feel quite bored, and I have no idea whether there is 1 more page waiting for me (in which case I would rather complete the survey) or 20 more pages (in which case I would rather quit).

Expand full comment

Part 2:

"This survey will take N minute" is a typical mind fallacy.

When you ask whether "doing things way better is possible", do you mean technical or political possibility? Because there are many things that could be easily improved by a dictator (which does not mean that the dictator would be likely to improve them, only that the possibility exists), but if you try to improve them now, the problem is that the current state benefits a few people who will strongly oppose the change.

"Are all politicians liars" compared to what baseline? Because most neurotypicals are liars.

Sometimes the answer scale is visually asymmetric, because the word "disagree" is longer than the word "agree", and the columns adjust accordingly. Not sure whether there is a simple way to help OCD users.

Expand full comment

Is doing all the surveys (ignoring the ones aimed at people you aren't) meant to be some sort of great feat? It seemed easy and simple to me.

Expand full comment

I so wanted to complete the quality of life survey, but found the instructions incomprehensible.

Expand full comment

I felt like I could figure them out, but I didn't feel like the time investment would pay off. So, I skipped it.

My guess is that the survey was really an IQ test in disguise.

Expand full comment

I'd be curious to find out what that "optical illusion" survey was *actually* testing for. Clearly, the "illusions" are supposed to distract us from something... but from what ?

Expand full comment

[SPOILER]: I looked away from the cross a few times because my eyes just automatically moved, and I noticed that some of the "uniform" visual fields were not actually uniform (e.g, the pentagons one sometimes had squares on the edges). I presume that is part of the illusion - perhaps the test is how long it takes you to notice this?

Expand full comment

SPOILER: Sometimes there are no differences between center and periphery, so that you should perceive uniformity (and press the button) as soon as the fade-in of the perfiphery completed and sometimes there are differences between center an periphery that seem to dissipate over time so typically after some seconds you perceive uniformity although it aint there (and only then press the button). We call that perceptual filling-in and by calculating the average time between complete fade-in and reported uniformity and comparing this time between illusionary and control trials one can infer how long this perceptual filling-in needs to take effect. We are especially interessted in whether this effect on low-level perception is dependent on affect and mood. Thanks for helping with this by participating!

Expand full comment

Whoever wrote Metaphysics, your question about intersex conditions is *awful*. XXY/Klinefelter's is only "ambiguous between male and female" if you know absolutely nothing about sex chromosomes or their aneuploidies. I recognize this misconception is fairly common, but that doesn't make it not "so immediately dumb that I considered closing the tab instantly, because if all these questions were written with that understanding of their concepts then no meaningful results could possibly come from them".

Expand full comment

FYI, Qualtrics surveys are not accessible over Tor, so on the more sensitive topics you may lose some people who would only answer from behind an anonymizing proxy.

On the other hand, I suspect a goodly percentage of the survey-takers could actually be uniquely identified by the information requested in the #0 demographics survey (though perhaps not actually located by it.)

Expand full comment

For next year, it may be good to set a norm for Likert scale directions (i.e. either always have agree on left/disagree on right or vice versa). There's some variance across surveys and have realized a couple times that they were not set up the way I thought.

Expand full comment

Wow, I had a really hard time understanding what was being asked of me in the hypothetical health states questions. I'm not sure how I would write clearer instructions for it, because I'm not entirely sure I get it still. The instructions almost certainly need to be shorter.

Expand full comment
founding

Nothing better demonstrates the questionable utility of surveys than filling one out!

Expand full comment
founding

I'm assuming #2 is for people with knowledge of their blood relations.

Expand full comment

For me, the optical illusion test took a little over 30 minutes, plus breaks for eyestrain. Regardless of what some commenters suggest, there IS an end to it, it's not testing just how willing you are to keep doing a repetitive task. However, it does save the data even if you don't have the time to complete it.

Expand full comment

I wish that the optical illusions survey (#21) had a button for "faded in completely, but I can still see/feel the rectangle border, so not an uniform image". Because in that case, you always have to wait while staring at a dot, and it requires a LOT of willpower. If I could just press A/B or something, that would make it a lot easier.

Also, I accidentally noticed that the rectangle border becomes more apparent when you slightly move your head while still keeping your eyes fixed on the red cross. I tried not to do it and be "good".

Expand full comment

(this is a test to see what my username is)

Expand full comment

The quality of life survey was nuts.

One, quite wordy and convoluted. Two, actually bloody hard! Had to think quite a lot about what appropriate answers might look like. Three, my priorities seemed to differ wildly from the author's.

Really looking forward to reading the results, the goal was one of those Basilisk extremes, monster cyborgs that no normal human or society would ever sign up for (hopefully), but there's more insight in it beyond the original idea.

Expand full comment

Well, I spent nearly four hours on Metaphysics, and then after submitting, reopened the test to see new questions about... fan conventions.

I shan't be answering those!

Expand full comment

I wonder how many other outliers are like me, and simply high school graduates, and what they're like. Whether they're all grumpy and cynical like life beat into me, or if I'm just an outlier among outliers. Or maybe they all have other mental illnesses like ADHD or other heritable disorders. It's really difficult to find outlier-based surveys where I might glean some insights into "my demographic", as generally "high school and some college" results are lumped together into "poor and ignorant" instead of "chronically curious but unable to pursue academia". Will you be doing a separate analysis of such persons? Are there even enough of them? Please do, I would like to know just to see if my psychology correlates to this or other factors.

Expand full comment

Oh we're here all right and while I can only speak for myself I've done most of the surveys at this point. *Hopefully* there are enough of us for some sort of analysis! FWIW I like to say that my cynicism is frequently outrun by the latest new absurdity and I, too, would be interested to see what kind of things would stick out with the HS grad demographic here--I'll freely admit that I have a few suspicions myself on that front.

Expand full comment

I started two surveys and had to quit both. My (undiagnosed) ASD doesn't allow answering questions where I know my answer will be interpreted incorrectly.

Expand full comment
founding

For whomever is responsible for survey #3, you wrote at the bottom of the survey:

> It would be very helpful if someone could suggest a way I could randomly generate hypothetical health states in an online survey (i.e. randomly complete the five possible states in their three possible levels). I understand enough 101-level coding to write a program to do this on my local machine, but I don't know how / where I could host it in a way that would be suitable for this sort of survey.

Basically, I'd generally suggest creating a web application and hosting it on (deploying it to) a VPS – a virtual private server. There are some other possibilities for what's effectively free hosting too, and you _could_ provide public Internet access to a program running on your own computer (and for free), but a VPS is the most general way to host a program publicly. (But keep in mind that this can be arbitrarily hard – the Internet is very much a 'densely populated ecosystem' and a publicly accessible server will be 'under attack' almost immediately after accepting incoming public Internet traffic. I wouldn't overly worry if it's only going to be live for a relatively short period tho, e.g. a few weeks, or even a month or two.)

I'm open to answering any other related questions here too.

Expand full comment

The metaphysics survey having "option 2" and "option 1" in that order is really aggravating. It might be random don't make me guess how to enter a right answer.

Expand full comment

In the optical illusion survey, I often found that the images looked uniform *before* I was allowed to press the space bar. I fear that this design choice (to only allow perceived uniformity after a certain time) may significantly impact your results.

Expand full comment

I also noticed that, but I've no idea whether or not it might affect the results!

Expand full comment

Button press was only allowed after the periphery was completely faded-in. So if you pressed before, you perceived a uniform field were there was none, which is certainly interesting, but not completely what we wanted to measure...

Expand full comment

Thank you. I suppose my comment that it may impact your results is premature without knowing your exact research goals. I assumed it had to do with measuring the accuracy of peripheral vision. In that case, perceiving uniformity where there is none would be an interesting data point.

Good luck with your research!

Expand full comment

Thanks for participating though!

Expand full comment
founding

I just finished all of them except #21.

To beat the dead horse I mentioned in a previous comment on this post, I found myself, SO many times, wishing for another option/answer to a survey question, or even just an 'other' option/answer with a text box, for at least _some_ way to escape the provided 'framing' and challenge the assumptions seemingly baked into to the questions! Having now just been subjected to that, over and over too, I hope I can remember that feeling every time I ever read or hear about any survey results ever again.

Expand full comment

Skipping #6 as I don't quite see how to answer it if I am not regularly measuring my weight.

Expand full comment

#21 takes me to the depression questionnaire.

Expand full comment

Rough showing for Christian Democrat political views. It was one of the dominant political parties and ideologies for multiple European countries throughout the 20th century. It is still in power in many countries today - Merkel is a Christian Democrat, for goodness sake.

But for this survey, that viewpoint does not exist. You can be a U.S. Democrat, or a Scandinavian socialist, but a Christian Democrat - nope, not on the table.

This is perhaps reflective of the readership and survey makers, but is not progress for the world in my view.

Expand full comment

I was 80% through the autogynephilia survey when I clicked on "Previous" to check something in a previous page and was suddenly back at the first page with all fields empty. Was my data never saved at all and I'd have to start from the beginning, or was my partial answer saved and I can just enter my id again, maybe put this in the first other field, and continue where I left?

Expand full comment

Woot! I took all the surveys that were relevant to me. Please feel concern for me!

Expand full comment

Me too! I even took all the surveys that were not relevant to me at all (which happens to be all of the targeted ones) as well. Also, please praise me.

Expand full comment

duck_master, you are smart and considerate and easily the best looking person in the room. I can just tell these things.

Expand full comment

An interesting thing to measure would be, for each user ID, how many questions they answered: there might be some correlations between that and other things.

Expand full comment

+1 for "I tried the optical illusion, gave up after a while".

Pity, I am really into optical illusions, but having no way to "stop now" or knowing how long it last, no thank you.

Having them one per page is also pretty bad UX...

Expand full comment

i took all the relevant surveys please praise me

Expand full comment

On my phone, for questions like the rhyming survey, the higher values are hidden - I see only options 0, 1, 2 in the rhyming survey, I need to be aware that I can scroll to higher values, or all answers will end up 2 or less. Maybe, something to control for?

*Redoing the survey*

Expand full comment

Can't give the Curium rhyme a 5 rating in the rhyming survey. Maybe an issue on my mobile phone.

Seems to work for all other lines

Expand full comment

The Survey #19 is a bit weird with how the responses are presented. For the first part of the survey "often" is on the right and "never" is on the left. So the more that statement applies to you the more you select your answer on the right.

After about half the survey it changes and "I agree with this" is on the left and "I disagree" is on the right.

I had to correct a bunch of answers because I caught it in time but I could imagine that some people answer the second part of the survey with the opposite to what they actually mean.

Expand full comment
founding

I took the 19 non-targeted surveys, 1 targeted survey that I was the target of, and accidentally 1 targeted survey I was not the target of by going in order and not noticing it was targeted/not reading the parenthetical after the link. Oops. There's at least 1 incorrectly targeted response to the Rhyme survey.

Expand full comment