745 Comments
Comment deleted
Expand full comment

I agree, but just so you know, it saves your responses even if you don't finish. For me, it took about thirty minutes including breaks for eyestrain.

Expand full comment
Comment deleted
Expand full comment
Comment deleted
Expand full comment

I read it twice and was still totally confused.

Expand full comment

I believe you can still skip the questions, even though the radio buttons may not let you unselect after you picked an aswer at random

Expand full comment

I'm half Japanese, quarter French, and quarter German. How should I answer the race question?

Expand full comment

Also, the profession question. I don't work or study anything at all.

Expand full comment

"Have you completed your formal education?" I'm a college dropout. I'd like to go back someday, but have no current plans.

Expand full comment

I'd say "no" to that one, I guess. I'm a drop-out with no desire/plan to back, so I said "yes."

Expand full comment

It feels weird to answer that way because it makes basically no actual difference either way. It's just something I tell myself.

Expand full comment

Ah, but are not the stories we tell ourselves amongst those most important?

Expand full comment

No, I don't think so. Not if they have no bearing on reality.

Expand full comment

The standard way to do this is "mark all that apply", disappointing that Scott's using outdated methodology.

Expand full comment

"Mark all that apply" is useful if you want humans to visually inspect the results, but makes things difficult if you want computers to analyze the results. With answers that are either yes/no, or a number, you can do a regression, or logistic regression, factor analysis, PCA, or lots of other powerful numerical algorithms. If you've got a bunch of boxes that can be checked or not checked, with no known relationship between those boxes, what are you going to do with it?

Expand full comment

Personally, I barely managed to resist the temptation to answer "Other: 100 yard dash"

Expand full comment

If you want to give those answers in excrutiating, painful detail, you can totally answer the optional demographic questions on the COVID-19 Impact survey!

Expand full comment

There being other questions on other surveys doesn't change the fact that there are these questions to answer on this survey.

Expand full comment

What are your feelings on the Alsace-Lorraine question? 😉

Expand full comment

My feeling is that the Schleswig-Holstein Question is the more interesting one.

Expand full comment

Was that on a different survey? I don't recall it.

Expand full comment

You said you are quarter french/quarter German, she made a joke about a question that divides these nationalities.

Expand full comment

Ah. Well, just goes to show how little connection I have with my heritage...

Expand full comment

White

Expand full comment

It's a bit late for that, for a more modern version "What are your feelings about the EU?" could get you some nice answers.

Expand full comment

It's never too late for this type of thing… and historically Taunton is a part of Minehead, I'd have you know!

Expand full comment

Personally, I skipped that answer. To me as a German, anything about race is super scary and weird, because it immediately evokes my countries history.

TLDR: Just skip it, I would say it's a weird US thing

Expand full comment

You shouldn't skip it just because it seems weird. Rather, all the more reason to investigate.

Expand full comment

Let me be more clear: I'm disgusted by the way Americans still file data on and talk about race like it's the 1940s. If you want to talk ethnicity or social standing that's fine, but everything else should not be in data like this. If the German federal state would ask for race data in, say, passports, the public outcry would be massive.

"more reason to investigate" - no, pretty uncool since nazi times.

I am aware that critical race theory exists, and how it's important in the US etc, but I'd rather not import the US notions of race into my own country nor thinking.

Expand full comment

Huh, that's a really interesting cultural difference. (I'm from the US.) If you don't mind me asking you to elaborate: Do most Germans consider themselves "colorblind" / think racism is a thing of the past? Or do most people generally feel like racism is still an issue in modern German society, but that explicitly categorizing race is part of the problem?

Expand full comment

Should have noticed this when these were first posted, but weaknesses in the demographic survey:

- No multiracial

- No Native America or Pacific Islander

- Hispanic is not a race, though I selected it (I am multiracial White and Native American)

- I have no clue whether people within two generations of my family have been diagnosed with specific mental illnesses and I'm not sure how widely that type of thing gets discussed outside of immediate family

- Was the above intended to only include direct ancestors? Because I took it to include siblings, cousins, aunts, and uncles

- You gave us the ability to say we're not interested in politics and not registered with a party, but did not give the ability to indicate we don't feel any particular allegiance to any of the listed named ideologies, either

Expand full comment

I don't understand much about how the US thinks about their races so forgive me for my ignorance but if "Hispanic" is not a race where do you put people coming from Latin/Hispanic America?

Expand full comment

By their race. White, black, mixed...

Expand full comment

Those are skin colors, not races.

Expand full comment

As a US person I think the Hispanic/non-Hispanic scale is gesturing at indigenous ancestry from different regions, and also at language, but then also at situations of general "otherness." The scale doesn't make sense to many people in the US either. Spanish and British colonialism worked out differently. British did not create the "mixed race Indian and something else" classes so that even if multiracial people with Indian ancestry existed they were subsumed into white or black. Spanish colonialism did create the category of "mixed race Indian and something else" and so the categories don't map neatly from one region to another.

Expand full comment

My impression was that "white" meant "mostly european ancestry", and thus hispanic was useful because it meant "mostly central and south American ancestry", even if those tend to be more mixed than Europe. The thing is, I think two things are mixed in the USA definition of race: the genetic origin of people, and how people are categorized into groups by other people based on their appearance, culture, the way they act. The first part is the "medical" part of the definition of race, and is useful in cases like sickle cell disease. The second part is the "sociological" definition of race, and is useful in cases of discrimination, racism, and studying society in general. The way both are mixed strikes me as unhelpful.

Expand full comment

As Marginalia said, the "Hispanic" category is cultural, an artifact of the Spanish empire creating a distinct mestizo identity and subsequent countries, most notably Mexico, explicitly embracing mestizo identity as their national identity. This is quite different from the one-drop rule of the United States, but just like the United States, the people populating other American countries, including Latin American countries, were either natives or they came from somewhere else, mostly colonizers from Europe and slaves from Africa, and their races are the same as the races of the people in the United States. The biggest difference is there are far more people of partial or majority native descent.

Expand full comment

Most official documents (employment applications and the like) have one question for "race" with a range of races, and a separate question for "ethnicity", with the two options of "Hispanic/Latino" or "Not Hispanic/Latino". As detailed in this presentation on the 2020 census: https://www2.census.gov/about/training-workshops/2020/2020-02-19-pop-presentation.pdf this dates back to the 1970 census, when the Office of Management and Budget defined "ethnicity" as a technical term referring solely to whether or not someone is Hispanic/Latino. Apparently, research has shown that integrating Hispanic-or-not into the "race" question is better design....but the 1997 OMB standards require that the question be asked separately. All the employment and university applications I've filled out follow this pattern, likely for similar legal reasons.

Expand full comment

What's with all the gender Ideology stuff in the 0: General Demographic Information survey?

At that I'm out ⛔

It looks more like an idealogical purity test than a demographic survey.

Sex Not Gender ♂️♀️

Expand full comment
Comment deleted
Expand full comment

I ignored my true beliefs on the matter and ticked the cis box, but my vast amounts of privilege makes this kind of lie very easy for me to bear.

Expand full comment
author

That's copied from my last survey demographic section, but I had it in there for a few reasons:

1. One of the things I used the survey for was to investigate what things are due to biological sex vs. self-identified gender. Without distinguishing between those, that would be impossible. I'm not going to prevent myself from investigating important questions just to avoid offending people. See https://slatestarcodex.com/2017/06/28/why-are-transgender-people-immune-to-optical-illusions/ for an example of some research I did with those questions.

2. Without that distinction, transgender readers might be confused about what answers to give, or people interpreting the survey might be confused about what their answers meant. Regardless of your ideology it's important to make sure you're communicating clearly with your subjects and that their answers mean what you think they mean.

3. Past research has suggested that about 3-4% of ACX readers are transgender, which is significantly overrepresented compared to the general population. I'm interested in tracking that and investigating why.

Expand full comment
Comment deleted
Expand full comment

Interesting hypothesis! Let's see how the survey data pans out--I think there was a question in one of em about autism iirc?

Expand full comment

Scott, I appreciate your taking the time to respond to my rather brief message.

⛔ I'm not in the least offended but I do object to the normalisation of a particular socio-political / idealogical framework as the normative view of human categories. That I most certainly do object to. ⛔

If the purpose of the questionaire was to determin the embeddedness of a particular Ideology in the readership then fine, we'd all know the purpose and could continue on that basis but this does not seem to be the case.

If that were the case then we should have a control group using normative categories to compare and contrast against and we don't have that, so again I question the appropriateness of it's inclusion.

And so to restate my position, and with regret, I do not feel I can contribute to a ideo politically compromised questionaire without providing tacit approval to its ideo political contents.

Expand full comment
author

"If that were the case then we should have a control group using normative categories to compare and contrast against and we don't have that, so again I question the appropriateness of it's inclusion."

I'm confused - what would that mean?

Suppose the readership is 100 cis men, 100 cis women, 10 trans men, and 10 trans women.

You can already tell that if you just ask "what is your gender?" without allowing anything about trans as an option, 110 of them will say male and 110 will say female.

Can you explain more how a control group would work?

Expand full comment

Once again thanks for replying.

My objection ⛔ is to the highly partisan language and idealogical basis of the foundational questions in this questionnaire.

You've chosen language only supported by ~8% of the american population, at most. And much less popular is other English speaking countries around the world.

https://hiddentribes.us/profiles/

Now a questionnaire on the relative support of Gender Ideaology would be a very worthwile thing to do in it's own right but mixing it's highly contentious and partisan language and ideas in has I believe biased this questionaire at it's outset and my participation in it would be to support these highly divisive idealogical ideas, which I will not support. ⛔

I'm sure such ideopolitical framing would help ward off any future New York Times hit pieces though 😕

Expand full comment

Again agree. Control of language is power of itself. See most of Orwell's work for further detail

Expand full comment

But on that particular contentious topic, I despair at finding ANY set of terminology that's "neutral"; EVERY set of words to describe anything related to sex or gender has become politically loaded and subject to endless fights, as seen in the edit history and discussion page of any Wikipedia article on the topic. (Myself, I don't like the "Assigned * at birth" construction... what, is there a Sorting Hat making the assigment? Biological sex is *observed* at birth.)

Expand full comment

I would be absolutely shocked if only 8% of the US population was supporting the idea of sex vs gender. You link specifies Progressive Activists, but i am most definitely not one and i support the idea.

Expand full comment

Yep. This commenter is clearly a crank

Expand full comment

I somewhat disagree with Gerd, but he/she/it/etc. has a point: it might be a good idea to include fallback survey options such as "I disagree with the very notion of gender as separate from sex", or something of the sort.

Expand full comment

Edit: as I'd mentioned above, it might be good to include some options such as "I'm just a man/just a woman", in addition to "cis/trans".

Expand full comment

Great idea. The people who select "just an X" are different from those who select "cis/trans."

Expand full comment

Good suggestion.

Expand full comment

You are presuming a normative framework, but demography is a descriptive framework.

If you believe that trans people don't exist, or that they shouldn't list, or that they are deluded perverts, or whatever, that is your normative perspective.

But a descriptive approach acknowledges that there are a whole bunch of people who believe gender and birth sex do not align in their case. Failing to account for that perspective makes your data worse. If you just ask "man/woman" when you really want to know birth sex, you get junk data. Even if a researcher wanted to use your normative categories, they couldn't trust their responses to properly disambiguate the data.

I am trying very hard to be charitable, but even if I agreed with your ideological position (and to be clear, I don't), I'm not sure how avoiding the question would allow you to get at the information you want.

Expand full comment

Surely, even from the point of view that transsexuals are delusional, it is useful demographic information to know whether a given subject suffers from said delusion?

I don't see anything in the survey that assumes self-identified gender is correct - it doesn't ask "what gender *are* you", but "with what gender do you primarily identify".

Expand full comment

I think this is like objecting to research programs whose goal is to study some topic that isn't yet thoroughly understood.

Expand full comment

I think you are close to a very important insight which can allow you to better empathize with people around you. The thing you've experinced is exactly how being offended feels from inside. This rightful fury, after visualizing terrible consequences if the things would continue to go this way, this feeling of menace, of violation of someones freedoms and liberties, of being enforced some categories and narratives which you didn't consent to - is exactly the way one of these ~8% of the american population would feel reading your reply and for mostly the same reasons!

We may want to distinguish between this cluster of feelings from bitterness or resentment. Something a children may feel when their wish isn't immediately satisfied. Is it what you've meant when saying that you are not in the least offended - that you are not like this children, that your concerns are much more valid? I can agree there can be a meaningful distinction and if so, then virtue of precision motivates us to try to reshape the categories to grasp this distinction. But by exactly the same logic there is a meaningful distinction between sex and gender categories and we ougth to grasp it in order to be more precise.

It's sad that these topics are politicised. But we don't have to accept this narrative of politization. We can just try to talk about categories as precise as we can trying to figure correlationsand causality between them. Imagine how annoying and counterproductive it would be if question of not being offended was heavily politicised. If people would immediately and with straight face start to tell you that you are a deluded liar for claiming not to be offended when you clearly are, that words have meanings and you are just trying to smuggle your political agenda into the conversation.

Expand full comment

Many readers of this blog and takers of this survey, myself included to some extent, would see a *lack* of acknowledgement of the existence/validity of transgender identities as an objectionable ideological framework.

Expand full comment

Maybe you need a “This survey is shit button—I’m out” button to track how many people see things as starkly as Gerd that you just aren’t tracking. Might be useful to know.

Then again it’s also what the comments section is for to some extent.

Expand full comment

I'm a white hetero married Republican Mormon, and I'm perfectly fine with answering that question. I'm curious about the data.

Expand full comment

Also just probably outed myself in the survey data, huh. But that's OK. I'm not too worried about my answers.

Expand full comment

Yep, there can't be too many of those around here, even considering the likely positve correlation of those traits.

Expand full comment

He definitely didn't out himself.

Expand full comment

I agree that from one viewpoint, it's annoying. But there are many people out there who are very much of a different viewpoint, so for the sake of taking a survey, if someone wants to identify as non-binary trans masc panromantic soft femme, let 'em and let's get on with it. I'm not gonna fight with someone else over their weirdness if they don't fight with me over mine.

Expand full comment

Same. I am not a cis woman. I am a woman. The "cis" label implies a whole bunch of baggage that I am not going to hold.

Expand full comment

This doesn't make sense to me, because the "cis" label implies the explict *absence* of any such baggage. All that weird gender identity stuff I don't understand or relate to? Doesn't apply to me. Won't hold it. Therefore, cis. The fact that this is relevant to *other* people's identity doesn't make the label that distinguishes me as "not one of them" an indictment, or imply that I endorse the ideology or whatever.

It's like, at first I live in an all-white neighbourhood and I don't even know the word 'Caucasian'. Then a black family moves in and they say "We're African-American, you're Caucasian." If they claim that they're deserving of better treatment because of this distinction, I might disagree. But I don't fight against being labeled as different when we *are* different.

Expand full comment

There's another way of looking at this: the "cis" label implies that there can be something else than a cis woman. I think that's what C MN is referring to.

Expand full comment

It might socially imply that, but it doesn't logically imply it.

BUT ALSO: If God (or a sufficiently advanced alien) 'changed' your sex in every possible meaning of the word 'sex' that you personally subscribe to, would that not make you the opposite sex by your own definition?

Expand full comment

I'm not sure what you mean by "logically implying" here, but surely if you say "cis women", you're using "cis" as a modifier for "women" and thus saying that there are one or multiple "* women". If not you wouldn't use "cis" before women?

> If God (or a sufficiently advanced alien) 'changed' your sex in every possible meaning of the word 'sex' that you personally subscribe to, would that not make you the opposite sex by your own definition?

I think that would, but I don't see what you're getting at.

Expand full comment

1. Saying 'large elephant' doesn't imply that small elephants exist.

2. If god made man into a woman, they would not be a cis woman (cis meaning assigned at birth). You seemed to be saying it is not logically possible for there to be such a thing as a non-cis woman, but this is an example where you agree there could be.

If God made 1% of all men into women, wouldn't 'cis' become a useful and meaningful term that is fine to include on surveys where relevant?

Expand full comment

I agree completely -- the term "cis" does impose a certain worldview on the reader. I have no problem with it myself, but I can see how others might. A better survey design might look something like this:

"I identify as:

( ) Cis Man

( ) Cis Woman

( ) Trans Man

( ) Trans Woman

( ) Simply a Man

( ) Simply a Woman

( ) Non-binary

( ) Gender-fluid

( ) Other

Expand full comment

That's a really good way to put it.

Expand full comment

Excellent solution. I think it is important to allow the option of conscientiously objecting to the implications of politically loaded and often intrinsically normative terminology being introduced under the guise of being merely descriptive

Expand full comment

Yes, as it stood, I just wrote it under "other." I'm just a body! My experience of my own brain isn't gendered (unsure how I'd even know if it was).

Expand full comment

Better yet to simply ask in questionnaire #0

Male ()

Female ()

Prefer not to say ()

and put these more ideo political questions within their relevant Gender Identity sub questionnaire #17 for those that subscribe to those particular beliefs.

Everyone would have been catered to and nobody left out.

Expand full comment

Agreed. I've been taught survey writing at the college level and that's how I would have handled it. Keep biological sex as its own question, and then for the people who believe they have a "gender identity" or whatever, they can answer question(s) about that.

Expand full comment

But then you would get ambiguity--people who do believe that sex and gender are separate would not be clear whether they should put their biological sex or identified gender if there was only one question with only those categories.

Expand full comment

A more fitting analogy - from the perspective of the critics here - would be if a family claiming to be aliens moved into your neighborhood, and thenceforth you had to identify as a 'terrestrial' in surveys, which would implicitly affirm the idea that some of their neighbors are actually extraterrestrials, which might be irksome.

Expand full comment

I see your point, I'd also be irked if surveys started asking me if I identified as a flat-earther in the same way they asked me about my gender, if only because it suggests that the Overton window has shifted towards flat-earth in a way that's unacceptable to me. But then refusing to take the survey, or demanding that they *not* ask anyone whether they're a flat-earther, just seems like sticking my head in the sand. Flat-earthers exist, no matter how much I wish they don't. Taking that out on survey-takers is actively counterproductive.

Expand full comment

As I said above, a better survey design would have questions that capture this view; something along the lines of, "I'm just a normal person and I don't think aliens exist", as opposed to, "I'm terrestrial, implying that I'm not an alien, even though that is technically correct".

Expand full comment

I don't see how it implies that. I am definitely a terrestrial bipod primate and saying that doesn't imply any of the people in my neighborhood I'm reasonably sure are also human might actually be extraterrestrials. If it was the case that some small minority believed themselves to be extraterrestrials and this had interesting social and policy implications, though, it would be useful to capture that information in demographic surveys.

Expand full comment

Bad analogy.

You're talking about a disagreement in empirical facts - where they were born.

There's no disagreement on empirical facts in the trans debate (or at least, not among educated members of the debate on both sides.)

Objecting to someone saying empirical facts which aren't true is different from objecting to someone changing semantic definitions of words.

Expand full comment

The "what do you identify as?" language seems to make both this and the original question about as fact-based and non-controversial as they can possibly be. Trans-women/men, defined as "People who were born in a male/female body and currently say about themselves that they are actually women/men" are definitely a group who exist - a question about that is entirely appropriate. In the analogy "do you identify as terrestrial or alien?" is a perfectly valid question. The fact that you may think "those people who identify as aliens are complete loons who are just as terrestrial as me" is not captured by your replying "I identify as terrestrial" - but it doesn't have to be.

Expand full comment

Funny, because I think it's the very framing of "what do you identify as" that is problematic. By asking what one "identifies" with, the implication is that in so "identifying," one accepts, likes or is content with or at least tolerates, and/or finds some sense of meaning in said identification. Which is not at all the case for me, with respect to my biological femaleness -- i.e. I am and have always been a female, but I certainly had no choice about that, and I do NOT "identify" with "being a woman" or "womanhood" (not that I even know what "womanhood" is supposed to mean).

No one asks whether I "identify" with/as my actual demographic categories of height, age, ethnicity, citizenship, weight, eye color, or anything else. Those things are just facts, ones that you can't control and didn't have any choice about. In one COULD identify as their preferred weight category, surely there would be a whole lot of overweight people who "identify" as thin. Surely plenty of old people "identify with" being young.

I selected "cis woman" because I am heterosexual female and I'm not trans or nonbinary or whatever all the other categories are. But I certainly don't "identify" with or as a "woman" if that means some type of socially-prescribed but amorphously prescribed gender role or necessarily LIKE or feel comfortable as a woman. I just am one and it would never occur to me that there was anything I could do about it or that anyone would care how I felt about it, and the same way that there is nothing I can do about my height or age, and therefore how much I "identify" with those things is completely irrelevant.

I didn't have a problem with the survey question, and I think it's necessary to include a question that clarifies which respondents are trans, but I can see that there is a problem with "cis gender" in that presumes that a person believes in gender identity at all, or identifies as anything.

Expand full comment

It *is* interesting, because I think I probably have a very similar attitude towards womanhood/femininity to what you are describing - that is, yeah, I have a female body therefore I'm a woman (*shrug*). I'm deeply suspicious of the very concept of "feeling like a woman/man" - that is, I believe people when they say they experience life this way, I don't experience it myself, I don't really understand how you can be so sure that you "feel like" ... some any disparate group of people with their own individual feelings about things.

But I take the whole "identify as" question much more operationally - if someone says 'are you a man or a woman?' what do I say? And, yeah, I am saying "woman" because I'm not taking gender-orthodoxy seriously (which would presumably prescribe that I should say "non-binary") and solely working on the basis of physical body. But they didn't ask that!

Expand full comment

It occurs to me that Social Justice makes it seem plausible to be very concerned about how one is perceived demographically, even if one is generally in disagreement with social justice.

Expand full comment

It is necessary to talk about trans issues, and impossible to talk about them without making the cis / trans distinction. It's okay to redefine our terms within a specific discussion. Words were not handed down by God.

Expand full comment

It doesn't, though.

Like, if I said 'anyone with a nose that extends between 2 and 5 cm from their lip is a flim', and that describes you, then calling you a flim doesn't carry any particular baggage between the literal meaning.

Expand full comment

So, if you believe in sex not gender, what should an androgen insensitive person write? Given that they could likely appear female to anything but a genetic test, and live their life in that manner. As a corollary of that, have you ever had your chromosones tested? Do you know that you are in fact the chromosonal sex you are based on your phenotype?

Expand full comment

I mean, to take devil's advocate, first of all they could just believe in phenotypal sex, and in that model there would then not be a problem with having "other" or "intersex" as a category.

Expand full comment

This is what the "were you assigned at birth" language sorts out, doesn't it? An androgen insensitive genetic-XY person was almost certainly assigned female and had that on their birth certificate and so can say "was assigned female" - if they're happy with that, "cisgender woman" as in "gender identifies as woman, was assigned female at birth" is completely appropriate

Expand full comment

Had a good chuckle at this one because I realized I have no idea which direction you’re being triggered from

Expand full comment

"All the gender ideology stuff" being the single question about if you're trans or not? Unless it's been edited since you wrote this.

Expand full comment

Not him, but I personally dislike the "assigned at birth" phrasing. It connotes (in my opinion) an arbitrariness which isn't really accurate 90+% of the time. We say that (for instance) people are assigned to a specific customer support person. Or at work, we say that we were assigned a particular desk and workstation. In each case, it describes an executive decision made by management for how things will go in the future.

Picking which sex to put a birth certificate isn't like that. The doctor (or midwife or other responsible person) is making a judgement call about which typical sex binary the baby is closest to matching (which is occasionally a difficult choice, and sometimes made incorrectly.) But we don't typically use the word 'assigned' for a situation like this. We use the word 'categorize'.

I prefer the phrasing "how were you categorized at birth?"

(That said, 'assigned at birth' is a common usage and not generally worth fighting. I suspect the OP's actual issue is that it's an unambiguous tribal symbol for the blue tribe.)

Expand full comment

The "categorized" phrasing is probably more accurate, but the common understanding assumes that the categories are natural and objective (which for intersex people, they aren't). Assignment entails some arbitrariness, which is appropriate (not that gender assignment is totally arbitrary, just... a little. In some cases).

Expand full comment

I agree. Agreeing to be labelled as 'cisgendered' is acquiescing to a use of language that I don't want to. In the same way that someone (forget who) in response to the notion that one should be able to choose one's labelled gender (er 'woman' not 'trans-woman') said 'man' not 'cis-man'

Expand full comment

My opinion on this is that any holdouts will simply be co-opted into the agenda. You only want a category "woman" or "man"? Perfectly fine for the trans types who want everyone to agree that there is no difference at all - "a trans woman is a real woman", not "a trans woman is a trans woman", after all. A biological male who had surgery and is loading up on hormones will be the very same thing as a biological female under such categories, and the language-police you disdain will see that as a victory for them, not for you.

If I have to pick "cis" out of "are you cis, trans, other" then I'll do so because I do insist that yes, there is a difference between natally female and female via surgeries and hormones so for me there is a difference between "cis" and "trans" (and the "assigned at birth" language annoyed me as well, but again, I'll go with that rather than surrender the field).

Expand full comment

I acknowledge that (for example) 'man' and 'trans-man' as categories has it's problems, the migration to 'cis-man' & 'trans-man' meaning I am 'cis-man' is without my consent being sought or given

Expand full comment

I happen to think “don't categorize me without my consent” generalizes pretty poorly as a “true” rule—and you might be interested to know that that's one of the problems I run into when communicating in some other highly genderqueer-etc.-friendly (I almost want to say -centric even though they claim to theoretically not be) circles! That is, people there taking it on as a social reality defense: “how dare you put gender in things where it doesn't belong”, or “if you think of me as anything other than *just* my declared gender you're causing problems”, and especially “if you have reason to identify a variant/specific case of category X and then add *anything* to a term indicating ‘someone who isn't in that subcategory’ and use it on me, you're marginalizing me”. Observing the symmetrical position here is… interesting.

This is without saying that the pattern has no use or is harmful in all cases—only that it generalizes badly, and the badness is especially salient here, because a hard “you may only analyze me using ontologies I approve of” will escalate any attempt at comparative analysis between groups that don't share ontology preferences on the relevant information into “don't even try”.

Expand full comment

My impression of the trans community is that they do want to be treated differently from ordinary men or women.

For example, I am a cisgender man, who is also fat and has long hair (though not for much longer I fear). Back in my youth, strangers who only saw me from behind (thus unable to observe my beard) would address me as "ma'am" or some variant. Socially speaking, this is a minor faux pas at worst. However, if I were transgender, such an instance of misgendering would be treated as a severe insult, and potentially hate speech -- and my sense is that the modern transgender community would endorse this view.

Expand full comment

In my experience, they don't make a big deal out of honest mistakes. They'll just correct you.

Expand full comment

Should I start rejecting the label of 'democrat' because I think republicans shouldn't exist and don't want to acknowledge them as a legitimate political orientation?

Because I would love to start doing that, sounds like a lot of fun.

Expand full comment

More like rejecting the label of "democrat" because you don't have any political beliefs at all.

The people who object to "cisgender" don't believe "gender identity" exists (for them), so they object to the insinuation their biological body matches something that doesn't exist (for them).

Expand full comment

Thanks, your ideology relating to these topics will not be recorded.

Expand full comment

On the weight loss survey if I lost a lot of weight due to a untreated medical condition, (then gained that weight back when the condition was treated), I'm guessing it'd be more appropriate to answer "no" to "have you ever lost more than 20lbs under a year" question, even if the answer is technically "yes"?

... I'm guessing "have untreated Crohn's Disease and c. diff for 6-8 months" isn't the sort of hot weight loss tips the survey is interested in.

Expand full comment

I wasn't sure how to answer in relation to pregnancy either. I answered honestly, but I think it will seem inaccurate.

Expand full comment

I ran into the same issue. Pregnancy seemed not to have been anticipated by the responses available

Expand full comment

Same.

Expand full comment

Survey 21, optical illusions, would not open with the error "A problem occurred:

Couldn't find cookie for study result ID 31188. Are cookies allowed in your browser?"

Expand full comment

I experienced the same error on all of Chrome, Safari, and Firefox.

Expand full comment

Fine on Firefox on Android

Expand full comment

I got the same error in Firefox on both a Windows computer and an Iphone.

Expand full comment

Forgot to mention, I tried on Safari and Chrome on MacOS

Expand full comment
author

Weird, it works for me (though it's very long and annoying). I'll email the author.

Expand full comment

Worked, but way too long. I abandoned it

Expand full comment

Yeah, had to abandon it as well. Please remove from the list. Did a lot, but it was hard on my eyes, seemed repetative and there was no indication at all how many where left :-(

Expand full comment

I felt exactly the same way. Sorry, but I have kids wanting their supper, and I really need to know approximately how long this is going to take.

Expand full comment

Yes, they should at least indicate roughly how long it would last. I gave up as well and had annoying visual after-effects for a bit making me slightly woozy

Expand full comment

I got the same on Chrome.

Expand full comment

Maybe if anyone *can* see 'the test', *that* is the optical illusion!

Expand full comment

I now realize that the correct answer to the metaphysics survey was "what survey?"

Expand full comment

I'm getting the same problem on Chrome, including incognito Chrome.

Expand full comment

I'm one of the many for whom the same thing happened. Tried in Firefox on FreeBSD and Chrome on Windows. Adblocker enabled in both, which I guess might be relevant?

Expand full comment

Same issue in Firefox, with adblocker enabled.

Expand full comment

I tried on Chrome, Opera GX, Firefox, and *Microsoft Edge*, no luck :(

Expand full comment

You tried Edge? That's *real* desperation! 😁

And this comes from someone who clung on to using IE to the bitter end until finally they forced me off with "Edge now, no other options" and made me finally move to Chrome.

Expand full comment

On the other hand, I'm someone who NEVER used IE as my primary browser at any time; I stuck with Netscape 4 long after it was obsolete until Mozilla finally got stable enough to use as my main browser; then stuck with the Mozilla Suite/Seamonkey for a long time until finally switching to Firefox which I continue to use.

Expand full comment

Same, on my chrome copycat, aka "brave browser"

Expand full comment

Sorry, I think what happened here is that Scott posted the wrong link, since the one that I sent was a generic one that changes into a user specific one once you actually click one it. Now the study think that one very annoying person would try to do the study very often :)

With this link, it should work:

https://jatos.mindprobe.eu/publix/1521/start?batchId=1702&generalMultiple

Expand full comment

Holy moly is your survey long. I’d have appreciated some kind of progress meter telling me how far through it I was.

Expand full comment

Yes, it´s an experiment rather than a survey, those are annoyingly long most of the times (since we need a lot of trials to account for the high variability in responses). Sorry for that, I´ll try to insert a progress bar in between trials.

Thanks for perservering anyhow!

Expand full comment

I was about 30 minutes into the survey with no end in sight, when I quit. Since I didn't make it to the end, I couldn't give feedback/ask about the study, is there any way I can do that, that is not a comment section?

Expand full comment

Yeah me too, I didn't have that much free time, I probably did about 20 images and then quit.

Expand full comment

They're not kidding that it's very long! I think you should give a break about half-way through to let people rest their eyes; I found that constant staring made the images start to 'burn into' my retinas and interfered with the following images.

It was interesting in that there was definitely one image that would not resolve into uniformity for me, while a few others did very quickly.

Expand full comment

Same here -- there was always that one problematic image that would keep popping up. Also, some of the images did cause an optical illusion for me, where the red cross would appear to float in front of the monitor.

Expand full comment

Finished the experiment. I'll note that with the exception of the blue, I almost always (90%+) pressed the space bar 3-4 times before it accepted it, presumably because the peripheral image wasn't officially fully loaded yet.

Maybe I'm the only one, but it seems that if you're using the timing of pressing the space bar for anything, then gathering the time before the outside is completely faded in might be more useful information. As it is, for almost all of my responses, you're going to be measuring where I was in the timing of hitting the space bar and then releasing and hitting it again, rather than anything more meaningful, because the image became uniform before I was allowed to hit the space bar.

I presume you've taken it yourself to see how it works in practice?

Expand full comment

+1, except less than 90% -- maybe 30%?

Evaluation should give special treatment to space bar presses that were very close to the end of the loading time, they could be the n-th repetition already.

Expand full comment

I also had the experience where pressing space did not always immediately move to the next screen. I was not sure if it was intentional to the experiment, a bug in the experiment, or an issue with my keyboard.

As it didn't suggest I should press it more than once, I didn't, with the exception of a few where, after pressing it the first time, it looked like it stopped matching the peripheral, and then the peripheral moved back in sync with the image again.

I'll note that because this effect, there were some points that I hit 'Q' because of looking away from the red cross _after_ hitting space.

Expand full comment

Yup, that works. I agree that if it won't mess up your experiment it would be good to have some sort of indication of progress. (Maybe just some words every now and then saying things like "You're about 1/4 of the way through.")

It seemed like some of the images appeared non-uniform, at least for some time, for a reason you may not have intended: after-image effects from the initial image. (Of course, maybe you totally did intend that, but I wanted to point it out in case not.) I would be entirely unsurprised if there were cases where the inner and outer regions genuinely matched perfectly but I didn't press the spacebar because the inner region still looked darker/brighter/less-blue/... because stuff was shown there sooner.

Expand full comment

Oh, while I'm making suggestions: as a reward at the end it would be nice if you linked to a list of what the images actually were, or a thing that lets you do the experiments without contributing to the statistics -- having tried my best to be Good all the way through and therefore _not_ see properly what was really going on at the edges, I'd have liked to see what it was I'd been missing or not missing in each case. Even better if you gave a bit of information about what effects you were looking for (perhaps with a note saying "please don't tell other people who might do the experiment later").

Expand full comment

I really wished for this as well.

Expand full comment

+1

Expand full comment

The webpage became unresponsive after about 3 images - the screen was just a red cross on a white background with nothing happening for 20+ seconds, and neither spacebar/Q did anything.

Expand full comment

Yeah, I got that, except I stared at that one red cross for over ten minutes before finally coming to the comments section to see if that was what was supposed to happen. :(

Expand full comment

How do I enter my key at that link? I didn't have a dark room handy for this survey so I haven't tried it yet.

Expand full comment

21 seems to be broken - "A problem occurred:

Couldn't find cookie for study result ID 31188. Are cookies allowed in your browser?"

Expand full comment

Have the same problem

Expand full comment

Same.

Expand full comment

Sorry, there was some link mix up.

With this link, it should work:

https://jatos.mindprobe.eu/publix/1521/start?batchId=1702&generalMultiple

Expand full comment

I don't know what to answer in Survey 0, Demographics, for the psychological conditions where I'm both guessing I might have it (though not formally diagnosed) and I know I have a family member who's been diagnosed.

@Scott, I'll wait several hours before completing the survey in case you want to clarify which I should answer?

Expand full comment
author

Just check that you're guessing you might have it.

Sorry those aren't multi-check boxes, Google Forms is really bad at letting you analyze multi-check boxes and I try to avoid them whenever possible.

Expand full comment

Tip for future surveys: Specify that the survey-takers should choose the first option that applies to them if multiple options apply.

Expand full comment

Survey 5 lists "Bupropion" and "Wellbutrin" separately. Is there some reason to do this, e.g. does Wellbutrin differ from generic bupropion in its effects?

Expand full comment

I was curious about this, as well.

Expand full comment

The survey should list the brand name as well as the generic. People may know the former but not the latter. I Googled them, like a good little boy/girl, but I shouldn't have had to.

Expand full comment

I thought it might be a technique to figure out how savy/informed a person is, but they don't seem to do that with other drugs.

Expand full comment

For family members with mental illness, what is considered family? Sister? Uncle? Cousin? Or strictly two generations of ancestors?

Expand full comment

More specifically, I assumed that only blood relatives counted, but it doesn't say so.

Expand full comment

first cousins are within two generations (shared grandparents), second cousins aren't

Expand full comment

Given that this is the summer, survey-takers in between (HS/university/grad school) and (university/grad school/fulltime employment) are especially likely to be in weird in-between states with respect to their occupations and residence statuses. Any preference on whether to answer as one's past or future self?

Expand full comment

If you are going HS->Uni, HS->HS or Uni->Uni I'd say you're still a student. If you're going Uni-> job but haven't started work, you're unemployed (sorry!)

Expand full comment

I answered as my future self, even though it'll take a bit for me to actually get to studying my chosen field. Mostly because my past self didn't really have a specific area of study.

Expand full comment

I have an identifier from a previous survey, I assume I should reuse that? The instructions don't explicitly say to use your old identifier if you have one!

Expand full comment
founding

Alert: Survey 2 doesn't ask for user ID

Expand full comment

That doesn't seem to be the only one; #7 doesn't either.

Expand full comment
author

Yeah, I told the survey makers this but they didn't seem interested in fixing it, so I guess they won't have that demographic information.

Expand full comment

Survey #15 also does not ask for ID.

Expand full comment

Trying to do the weight loss one but I can't answer whether I have ever lost more than 20 lbs in a year as the side-effect of a medication or for some other reason because the answer is both. This seems to assume that if you have ever lost more than 20 lbs in a year, it only happened once. I suspect that is not the case for a lot of people and certainly isn't for me (for me, it's mostly because of sports, but at least once, it was actually due to medication side effects).

Expand full comment

As a point of reference, if you pay attention to NBA basketball or NFL football where skinnier players bulk up to get strong in the offseason, but struggle to keep that on during the season, it really isn't uncommon to see the same guy lose 20 pounds in a year every year for 15 years in a row. For weight class athletes, they might lose 20 pounds 10 times in the *same* year.

Expand full comment

The instructions say, "If you lost a lot of weight more than once, pick he most extreme case."

Expand full comment

Sure, but how do you treat that when it's roughly the same amount repeatedly? The in-season, off-season cycle, or fighting-weight, training-weight, tends to involve gaining and losing the same amount over and over. No single event is necessarily more extreme than others.

Expand full comment

Survey 13 includes mescaline in the list of "psychedelic tryptamines", but mescaline is a phenethylamine, not a tryptamine. This makes it unclear whether people with experience with other psychedelic phenethylamines (2C-B, DOM, etc.) should include them in the "other" category.

Expand full comment

Likewise for LSD.

Expand full comment

I'm willing to give LSD a pass as an honorary tryptamine. It does contain the tryptamine skeleton inside its ring structure, so it can be analyzed as a highly substituted tryptamine where the substitutions include additional rings. It is also covered by the Shulgins in TiHKAL.

Expand full comment

Yes, I had the same concern. I left out any non-tryptamine psychedelics (excluding LSD and analogues if we want to be that fussy)

Expand full comment

Survey 19 has the polarity of the answers switch halfway through the survey with no notice (ie, 1 switches from “strongly disagree” to “strongly agree”). I suspect the answers after that point are going to be spoiled unless that’s fixed.

Expand full comment

I noticed that! I nearly went the wrong way round and I wonder if that was deliberate to make sure people hadn't just picked a number and were filling in by rote?

Expand full comment

I wondered if that might be intentional based on the survey content.

Expand full comment

It's good survey design to mix up the polarity question to question (and user to user, if you can get away with it). Unfortunate that it might come after a series of questions that don't do so, but such is life - hopefully it'll wash out with the randomized survey order.

Expand full comment

Why is that good survey design? Doesn't it increase the risk of people answering the opposite of what they mean?

Expand full comment

Granting that you don't want to establish an expectation and then flip it halfway through like that one did, it's nonetheless important to shuffle around the presentation to fight against directional biases in response. It's similar to how a good election ballot randomizes the candidate order *per ballot*, because we know that being first on the ballot is automatically worth a few percentage points.

Expand full comment

Ditto

Expand full comment

What's the deal with user ID? Where can I get one? Is it necessary?

Expand full comment

There's a link to a random ID generator in the first survey.

Expand full comment

What's to stop you from using a nonrandomized User ID? Why is a randomized one required?

Expand full comment

Nothing stops you and it isn't required. But a random one is better at maintaining anonymity and less likely to be picked by multiple people.

Expand full comment

For me, survey 21 says: A problem occurred:

Couldn't find cookie for study result ID 31188. Are cookies allowed in your browser?

I have cookies enabled.

Expand full comment
founding

Some commented elsewhere in this thread with a corrected link.

Expand full comment

I'm a bit unclear on what I should put down as my work status since I'm a student but am also working at a convenience store. I guess I'll put student?

Expand full comment

The digital literacy survey has some typos and grammatical errors.

Expand full comment

Completed all of the surveys that apply to me, other than the optical illusion survey which doesn't work. I consider this sufficient to boast/worry about, as per the post's suggestions

Expand full comment
author

Congratulations! (also, you're *fast*)

Expand full comment

I had to leave out the fun ones about psychedelics and being trans and so on, because I am Very, Very Boring And Dull.

Still, I hope I can boast/worry in company with my peers here!

Expand full comment

Same! We should get a medal.

Expand full comment

Wow! If you want to complete the last one also, try the fixed ink :)

https://jatos.mindprobe.eu/publix/1521/start?batchId=1702&generalMultiple

Expand full comment

Same (I did get the optical illusion one to work for me and I feel doubly proud to have sat all the way through that one.)

Expand full comment

Same here. When you do it over the course of few days, it's not much at all.

Expand full comment

I just took all of em (except the ones that don't apply to me) in a blatant bout of procrastination from packing!

Expand full comment

I find the political spectrum question tricky because I consider myself a centrist in my country but that would put me very much on the left in the US.

Expand full comment

The people asking questions about US politics will probably just exclude non-Americans anyway.

Expand full comment

I wonder - one thing that comes up in my conversations with Europeans is that while I'm a centrist in the US I would probably also be a centrist in, say Norway (which would put me much further objectively left) because their government is better run.

Expand full comment

Survey 1's probability section asks for answers from 0-100 in percent, but requires a numerical answer with no other characters (and also no bounds on possible answers). This creates a lot of practical ambiguity if some people revert to answering on a 0-1 scale, since having all one's answers be less than 1% is not unreasonable.

Expand full comment

That one is poor, but since it was phrased in "things happening TODAY" terms and since I don't think cryonics (as we have it in the past, now, and the near future) is anyway workable, I gave all those a resounding 0.

*Maybe* in 50 - ? years time, something that is feasible will come along, but honestly? Right now? I think it's the equivalent of sticking Auntie Gladys into the chest freezer after she's popped her clogs and hoping some day something can be done about it.

Expand full comment

Yup. I just answered those with a 0.000001.

There should be a 'technically possible, but not happening' button on those

Expand full comment

50 years is way too early. I am not convinced cryonics is even theoretically possible, let alone technologically achievable.

Expand full comment

I entered decibel answers, and they weren't rejected, so I assumed it was alright? But I did find the wording confusing, and wondered if I should round to 0 on the smaller numbers.

Expand full comment