199 Comments
Comment deleted
Expand full comment
Comment deleted
Expand full comment

"as fairly describe as" seems a mistake?

Expand full comment

As a child psychiatrist, I think this is important to keep in mind when we prescribe stimulants. At the same time, we're mostly focused on function and actual learning is probably more of a secondary or tertiary goal by the time we're seeing them.

Expand full comment

They should’ve let the kids loose on Wikipedia instead and found a way to measure which group learned more

Expand full comment

This just seems weird. I would expect the dopamine spike from the medication to improve immediate recall in everyone.

Expand full comment

Selective attention for the win!

In my experience as a teacher, paying attention to the content is good, but thinking deeply about something sparked by the content is even better. Surface level knowledge often doesn't get beyond the flash-card recall level. But deeply mulling over facts or drifting between them in a semiconscious search for patterns can create lasting updates to one's mental models even once the facts are forgotten. From the outside that can look like spacing out. I may or may not be able to ask the right questions to see if anything interesting happened on the inside.

On the other side of the desk, the dullest teacher trainings I get to sit through are often the best, as I can't help but think about how much better I could have taught it... which means I end up thinking quite deeply about that topic and experiencing various micro-epiphanies.

Expand full comment
Jul 6, 2022·edited Jul 6, 2022

In Artificial Neural Networks, there's literally a variable called 'Learning Rate'. This is a number, you can set it to anything you want. And indeed, the higher you set it, the faster the ANN learns, all else being equal! But also, the higher you set it, the lower the maximum skill level the ANN can attain before it reaches it's limit and plateaus.

Expand full comment

I've taught 4th grade for almost twenty years, and I've had students who I would have sworn were not paying the least attention and were not getting a thing I said...but who, it turns out, were getting a hell of a lot more than I thought they were and who wound up doing pretty well. It's not always easy to judge such things by the obvious, superficial indicators.

Expand full comment

AI was brought up at the end, but maybe not the most relevant example. Many neural nets employ "dropout", where 30% or so of the neurons are turned off at random. This seems to help the network develop resiliency, and not depend too much on any particular set of neurons. To extend the metaphor to its speculative extreme, one could imagine that with all neurons paying perfect attention to the task at hand, you would be better behaved for sure (no neurons to misbehave with), and you might even perform better on immediate tests, but you might not "learn" in a resilient way.

Expand full comment
Jul 6, 2022·edited Jul 6, 2022

I don't think the implied premise of attention in class = learn more material is rock solid to begin with. We don't know what makes kids actually learn better, or at least we don't apply it in schools; instead they get an overworked 27 year old's heuristic approach to what they think teaching is supposed to be.

By the time they are in this study I think the boys have just developed learning styles that fit their strengths and weaknesses, ability to catch random detail but short attention spans. Having been in that situation (and been on and off Ritalin) I am unsurprised that the boys who continued to use the learning style they were experienced in kept up with those who were medically given different strengths they weren't used to.

I'm curious what would happen if you tried this with non adhd students. I would expect a stronger pro Ritalin effect as you'd be strengthening a preexisting strength rather than a weakness.

Expand full comment

I suspect you're right in the extreme case: there are certainly people you couldn't teach quantum mechanics to even given 20 years.

But that type of extreme case shadows a *lot*. In practice, we don't really care if we can teach someone something over truly long periods -- if someone can't understand a fairly wide topic in 1 year of poorly focused instruction (a classroom of 20+ students as compared to, say, a private tutor), we have no interest in teaching it to them. Given how long it took me to understand (for example) partial differential equations, I'm confident we lose a lot of potential this way. In other words, we rarely make anything like a serious effort to optimize training our meat computers in any task or subject area. Not sure that's relevant to stimulant function in the classroom, but if we ever work out an educational method that *isn't* inefficiently (human) labor intensive, it means we could see fairly dramatic gains.

More on topic, to echo other people here: attention is not a substitute for interest. I have paid attention to a lot of classroom instruction that I knew I had no interest in and was going to forget immediately because nothing had flipped the switch in my brain saying "this is actually worth knowing". My impression is that there are chemicals that might have something similar to that effect (among others), but they're mostly illegal drugs.

Expand full comment

Intuitively, I feel like the analogue of training data for a vocabulary word would be something like how many examples you've seen of the word being used. I don't think the *primary* way humans learn language is by memorizing formal definitions.

Expand full comment

"a little more quantum physics every day for twenty years, and eventually expect them to know as much as a smart person would after getting a four-year degree."

I think this would work (and does work - relatively poorer learners in schools at the moment still learn a lot of stuff by the end of school). But there's a condition. I teach English to children, and one of the problems I find that some seem to have run into is coping strategies. We get kids coming in at about age 10, who have been having English classes for four years (they start in 1st grade) and have at some points performed reasonably well on school tests, but they can't identify English words like "I" or "and". They seem to have developed (highly sophisticated!) strategies for giving correct answers on tests and in class, despite not knowing any of the stuff that we call "English".

Those strategies sometimes block any further learning. The student's goal is usually nothing more or less than to pass the test; if they have a strategy that sometimes/consistently enables them to do this, they will actively resist taking on new information that might disrupt their strategy.

I guess that means I agree with Scott that school is badly aligned, i.e. for a large number of kids, it never becomes obvious that getting good at the school subjects is the best way to get through the day; and they aren't given compelling other reasons to want to get good at school subjects. And a little bit of concentration probably doesn't make a big difference

But I feel like it should make a big difference over the longer term, because the kids will build up marginally more experience of having paid attention in class, heard something, used it in a test later, and gotten through the day a little easier.

Expand full comment

Could the answer lie in something like the lessons these kids were taking having been the same (i.e. using the same plan and resources, with teachers using the same strategies) and basically optimised for kids with ADHD? If so, perhaps the results just mean that Ritalin works *and* SEND teaching works (I've spent this year doing teacher training, and we keep being told that the ideal goal is for a kid with ADHD in your lesson to be able to learn as much as a neurotypical one, and it sounds like these lessons might just have reached that standard).

Expand full comment

Re the Ritalin study, which showed that attention improved but learning not so much when ADHD kids got Ritalin, how about the idea that engagement is necessary? When talking a test, students are usually trying to succeed, so they’re engaged. Typically, classwork is boring, so they’re not. I recently read of a study that showed that 1:1 tutoring gave better learning results than either of two classroom paradigms. Could it just be that 1:1 interaction is so much more engaging than being in a classroom? Perhaps in a 1:1 situation, Ritalin would help ADHD kids learn better. Or in other engaging situations, like, say, chess or video games or something.

Expand full comment

Hang on, I'm not following the logic here.

Solving math problems is one thing. Following rules is another, related, thing. Learning is another, and doesn't really seem all that closely related to either. You treat it as strange that improving the first two doesn't seem to affect the third. But really, why should it?

You say "Concerta's clearly doing something". And you're right. But why jump to the conclusion that the thing it's doing is "making kids pay attention"? There are other things that could explain better student performance.

I am a math tutor, and I often find myself wishing that my students could become dumber at will. That when the time came to do arithmetic, they could shut off most of their brains and execute rules mechanically. Because frankly, most thoughts are simply obstacles or distractions when you're trying to do kid-level math. That's why a simple computer can out-calculate a brain that has vastly greater processing power.

My first thought, when I heard about the effects of Concerta, was that it might be doing something like that. Inflicting useful stupidity, desirable narrow-mindedness. Which might actually make people worse at learning, not better. A wandering mind is a problem when you're doing arithmetic, but I think it's a good quality in a student overall.

Expand full comment

This is entirely anecdotal evidence, but as a "gifted" child who got diagnosed with ADHD at age 27, I've always completely separated my ability to learn (processing+analyzing information) from my attention issues. I take an off-brand Concerta, and what it allows me to do is exert less effort to do things I otherwise get paralyzing executive dysfunction for. I.e. just about everything in my life that I'm not really interested in on a whim. Concerta is a tool that allows me to direct my attention better, and most importantly it helps massively with time regulation: so less "oh I can finish this task in 20 minutes because I couldn't make myself do it earlier" and more "ok I want to do this, I can do it at X hour, and allocate Y duration to it." This process is near impossible without the medication. As a child this translated to setting my alarm at 5am in the morning to fail to complete homework and ending up scribbling it in class before it was due.

As stated at the end of this piece, I feel like the major hurdle in these types of studies is defining significant variables that first qualify the ability to learn, separate from overall attention. But you very quickly get into extremely murky territory (and I'll admit I'm not super well read on this topic because I find filtering out my personal bias/experience extremely draining, a poor show of rationalism at work).

Expand full comment

Could it be that Ritalin is somewhat dumbing down the brains ability to process new information, which counteracts the increase in focus?

Expand full comment

> Something like this must be true if we assume that it takes a certain intelligence level to learn surgery - or quantum physics, or whatever. Otherwise you could get a very dumb person, keep teaching them a little more quantum physics every day for twenty years, and eventually expect them to know as much as a smart person would after getting a four-year degree. I’ve never heard of someone formally trying this, but I predict it wouldn’t work.

... Don't we all start off as very dumb people, and we keep learning a little more every day for many years until we can do the complicated stuff?

Kids with low IQ hit the same milestones as kids with high IQ, just slower. Why would this be different in adult life?

Expand full comment

in regard to surgery students, maybe your brain just gets numb. at first you're excited - your brain is titillated. new, fun stuff is gobbled up. after four years it's just another serving of cottage pie.

to bring it to the classroom, not paying attention is an act to keep from getting bored. which is to say when you are paying attention you're paying attention.

Expand full comment

As a language teacher, I can confirm from long experience that not all words, even those considered at the same 'level' are equally difficult to retain. Nor can you predict with much confidence whether a given student will retain a given word based on how other students do.

From a language research perspective, I do not put much faith in results from vocabulary teaching unless those words are guaranteed to be unknown to the subject. This was obviously not possible here, but the fact that these words were part of the standard curriculum and were relatively common English words means there will be significant exposure effects.

(Another possible explanation as a former student and teacher trainer: many teachers are just not very effective, for various reasons.)

Expand full comment

I'm not sure if it's relevant at all, and this is just my personal experience so only anecdotal. I knew just a little German as my third language (2 hours per week between 6th and 10th grades). When I moved to Netherlands and started learning Dutch it replaced my German instead of becoming a separate language that I know. When I try to learn other languages they don't replace anything else, but since German and Dutch are more or less cousin languages my brain is only able to keep one among them. Now I still remember some German when I push myself, but it's a bit locked behind Dutch.

Expand full comment

"So maybe even among seemingly similar words, some are arbitrarily easier to remember than others..."

Here is some work on the intrinsic differences in memorability of words:

https://www.nature.com/articles/s41562-020-0901-2

Expand full comment

The obvious answer is that learning requires more than just paying attention.

Going a bit deeper, I suspect that learning - or at least learning in a useful way - is more about patterns and linking each new bit of information to existing bits you already know. That might be innate, or it might be possible to teach this. Rather than simply force someone to rewrite the laws of quantum physics ten thousand times while on Concerta, it probably makes more sense to figure out the underlying patterns and ideas, and focus on teaching them that.

Expand full comment
Jul 6, 2022·edited Jul 6, 2022

For certain subjects, I believe that homework matters a great deal. Because this is contrary to everything that the primary-education establishment has been saying for a while, I am fairly confident that I am correct. From my elementary school days learning multiplication tables to my undergraduate days learning organic chemistry, there was no substitute for simple drill: doing problems over and over until the concept stuck. You could show me the Friedel–Crafts acylation on the blackboard as much as you want, but I didn't fully get it until I went to the library and drew the arrow-pushing diagrams myself.

Prospective lawyers have two months between graduation and the bar exam. Generally they spend several weeks of that time attending bar prep courses. (This is because law school does little to prepare you for the bar exam; what this says about legal education, licensing, and testing is a separate debate). The most well-known courses have live instruction and charge $2,000-$3,000. From experience of two decades of education, I knew that the important thing for me was to do the thing that I would be tested on. And I didn't need to spend $2,499 to do that; I just needed to buy used practice tests from eBay and work through the multiple-choice questions.

So I'm inclined to believe that homework is where learning takes place. This seems consistent with the objectively poor results (50-75%) on these summer-school tests; I think it's a safe assumption that they were not assigned homework.

Expand full comment

Slight pushback against this: I'm very experienced in not understanding a topic at first, and then many months (or years) later, suddenly getting it because my method of inference has changed for the better. Although the effects of "raw ability" are real, they're not the full explanation I think

Expand full comment

Re vocabulary: I'd definitely expect that some words will be easier than others *for a given person*: words you can connect to things you regularly think about will be easier to remember.

(Btw cramming lists of words must be the most boring way to learn a language, and I suspect one of the least effective ways.)

Re: surgeon residents: An explanation is that they regularly use some of the things that are on the test, and by the second year they'll remember those. Other things they rarely or never use, and among that they'll only know whatever they remember from their earlier education (and, if the test has stakes, whatever they can cram shortly before the test, which they'll forget shortly afterwards).

Expand full comment

Your comment assumes learning is binary - you've taken a fact on board, or not. But both fact-learning and process-learning are probably more complicated than that. Educators talk about the process of assimilating learning in stages, though, sometimes described as acquisition, competence, mastery. Do you know it only in the first context? Can you do it easily and automatically? Can you apply it in a new context if someone calls it up? How about it you have to think yourself of calling it up from among all the facts or processes in your head (Homer!)? It seems likely to me that kids (learners) might be better at getting started by taking on stage one with better attention, but that they will still need similar hours or repetitions or opportunities to move up the ladder to mastery.

Expand full comment
Jul 6, 2022·edited Jul 6, 2022

One parallel I would draw to AI is that learning and intelligence are functions of compression: We don't remember all the facts we learned but hopefully they fed into a deeper more generalised understanding of the subject area. So forgetting specifics is part of the same process that also synthesises a more general understanding.

If I read lots of medieval history books soon enough I will forget the names of most people involved, the dates and many more specifics. But I suppose the more facts I learned and forgot the more nuanced the retained understanding of the general structure of the thing would become - I would become closer to grok the era. But this kind of understanding is much harder to evaluate based on taking a test so it might not bear out in an average study.

For language learning, obviously you do need to remember the words. But presumably a summer camp is too short to allow the brain to create the internal compressed language model necessary in order to speak a language. But only once such a model is in place learning vocabulary becomes something other than rote learning of random unconnected facts?

Expand full comment

Very rarely I use a small dose of Ritalin (apparently 1/4th of what some people in the US give to children, wtf is wrong with you) to get a lot of boring things done. The key is to create an environment in which there is a continuous stream of things to do and enter a flow state. It's best to close IMs and slack, unless you want to end up writing an essay of your thoughts to an unsuspecting friend.

On the other hand, I can't think of a _less_ flow inducing environment than a classroom, where there's some kind of boring lecture, no research materials other than maybe a textbook - if you don't understand, where are you supposed to find information that will help you understand? Talking with your classmate or drawing or doing literally anything else is more absorbing for a mind that wants to do something, anything, ASAP.

The bottom line is, modern education sucks, drugs won't fix it, go full autodidact and leave humanity behind.

Expand full comment

I’m one of those kids who was diagnosed with ADD in the 90s, but whose parents looked at her good grades and decided not to medicate. I always did great on tests, but I feel like test taking is not a great way to determine if these drugs are useful. I often wonder if I could have gone through childhood and adolescence NOT constantly forgetting things.

I’d write a great book report, then leave it at home and turn it in late. My mother was constantly dropping off things I’d left behind. She didn’t really connect this to ADD, or see it as something Ritalin could have helped. By high school, being scatterbrained had become part of my identity, and my nickname was “Airhead”. I had great test scores the whole time, but I felt helpless and incompetent. I’m concerned that leaning too hard on test scores might make people miss the big picture for their own kids.

Not that any of this has dinged me too hard in life; I have an interesting career and a wonderful partner and kids. Whatever coping strategies or mental hacks I’ve developed in 30 years are mostly doing the job. But sometimes I wonder if I’ve been rolling the ball uphill this whole time, and if a little Ritalin might still be a good idea. But “on paper” I’m doing well, and I can’t make up my mind whether or not trying medication would be a good idea. I’d like to see more research that looks at general well-being, since measurable performance isn’t where I’d expect to see an improvement.

Expand full comment

Has anyone tested whether or not if you have huge cash prizes for success in school, kids would do better? It’s not hard to imagine that what’s happening is this stuff just doesn’t seem that important to them.

It seems like in almost any field, you need increasing amounts of money to power experiments.

Expand full comment

Fascinating analysis and speculation.

I had the unexpected experience of teaching high school math for about 8 years, with at least 99.9% math ability myself. During that period I also realized I am high on the spectrum.

This is how it seemed to me. I describe most people as learning in parallel columns. They did not seem to automatically mentally search for connections between new information and everything they already knew that could possibly relate to it. That would be an untidy node network. But it greatly reduces the amount of separate information one has to store. Information has meaning. It is not a set of facts. This is consistent with studies I have read that people who are good at math use less mental effort, not more, to accomplish more than people who are weak at math.

People who establish a new mental category for every topic and master or fail to master it do not seem to be able to think the other way, and that other way can not be taught IMO.

This is totally black box, inferential, anecdotal observation. I doubt I have more neurons. But I seem to have a different method of connecting them.

It was intriguing to attempt to work out ways to motivate students, and figure out what they were failing to see that seemed self-evident. How to unsee. I try to use analogous strategies for my own social blindness, with mixed success. (I did get some feedback that I was a caring teacher, encouraging, and students' test scores were higher than my colleagues'.)

Expand full comment

At a first reading, I want to challenge the entire model these researchers are using. I think it misunderstands what ADHD is actually like and what the actual issues are.

I've been on and off (currently off) stimulant meds for ADHD over the years. I was *constantly* accused of "not paying attention" in school, but that was simply untrue. I was hearing everything the teacher said and learning it just fine, even if I was also shredding paper with my fingers or literally reading a book during instruction. In fact, I absorbed the material *better* if I was fidgeting or reading a book during instruction, because having a second stimulus helps me focus. I was considered an underachiever due to constantly forgetting homework and seeming inattentive, despite also acing nearly all tests and getting perfect scores on standardized tests like the PSAT.

If I did struggle, it was always with forgetting to do things (did this study allow the kids to simply forget to do their homework?) or with an ADHD trait that non-ADHD people rarely understand; the "brain not available" effect. Having a stimulant to overcome "brain not available" - the frequent times when no amount of staring at the paper and willing myself to do the problem would work - would be very helpful for i.e. test-taking and especially helpful for homework. (Test-taking is exigent, ADHD brains love exigence, homework is not exigent.)

Expand full comment

Didn't Scott already do an article suggesting that years/hours of schooling make only a minimal difference to educational outcomes?

https://astralcodexten.substack.com/p/kids-can-recover-from-missing-even

Expand full comment

Great info here, but slides off the rails a little around rote memorization of vocabulary words from flashcards. Surely this is not (except for people with eidetic memories) a meaningful test of greater than short term vocabulary retention?

Memorizing definitions doesn't, IME, convey meaning adequately to spark retention. Like, if you have no familiarity with the sport in question, so no visual or domain knowledge to fit your understanding into, how will you conceptualize a flashcard definition of the word "regatta"? Why would you hang on to that knowledge once you had committed it to memory for purposes of taking a quiz?

Expand full comment

Human Scaling Hypothesis: Human learning capacity are largely determined by three factors in order of importance: mental capacity, diversity of experience, and time.

Expand full comment

My understanding of learning based on a *very* superficial review of the literature and my experience is that there are four kinds of memory: working verbal memory, working non-verbal memory, short-term memory, and long-term memory.

Working verbal memory is the most straightforward: Someone gives you a phone number and you rehearse it mentally until you get to the phone. You can hold a single-digit number of "pieces of information" this way, but can improve this by "chunking" multiple items into a single "piece of information." Once you hit your limit, new items will replace old items, but it's neither FIFO or FILO. Instead items tend to come off the list based on what your brain decides is most important according to some algorithm I don't understand.

Working non-verbal memory is basically just visual recall. I'm mostly aphantasiac, and my working non-verbal memory is essentially non-existent so it's very hard for me to understand this one. I know how intelligence assessments test it though: you'll be shown a picture, the picture will be taken away, and about 30 seconds later you'll be asked to describe it in as much detail as possible. Apparently some folks just refer to a mental image that is held in their brains. At any rate, nonverbal working memory also is limited in some way (I assume to a few "scenes") and items also get replaced.

Short-term memory I know very little about but it's essentially what lets you remember what you had for breakfast two or three days ago but not on May 23, 2003.

If your brain finds an item important, it will move it from short-term or working memory to long-term memory. I'm speculating here but for me that *only* works if there's a way to link it to something I already know and care about. This is how I have always learned new vocabulary: "legislative" and "legal" clearly have the same root, and both deal with laws, for instance. If I needed to learn Mandarin, I'd be in a lot of trouble initially, because there isn't a lot of verbal similarity so not a lot of nodes on my mind map to hang new language. Studies would seem to back me up here: When young your brain creates new long-term memory super easily, but with time it gets harder to add stuff as neuroplasticity decreases. And that's credited as the reason *most* folks struggle to learn new languages.

Anyway, based on that model of learning and literature on stimulants, I'd expect that Concerta would improve working memory but would not necessarily have any impact at all on converting working memory to long-term memory. This seems extremely consistent with that hypothesis.

I'm not...sure this means that putting students on Concerta is meaningless. It just means that school teaches different skills than long-term recall. It's very very clear that most students, even those that are bright and do well, don't come out of school with a life-long understanding of physics *and* history. It's also not clear what giving bright adults a long-term understanding of physics *and* history would accomplish. Instead, school seems to be focused on 1) exposing students to topics that they might find interesting enough to dedicate long-term memory space to, and 2) teaching students to do boring stuff they don't care about. I'll mention again that "doing boring stuff you don't care about" is an underrated skill, and one folks with ADHD struggle with in particular. That said, it might also kill their spirits so *shrug* who knows.

Expand full comment

With vocabulary in particular, I would expect the learning has almost nothing to do with what is in the classroom and everything to do with how much the kids are hearing and using the words outside. After all, unless a word really nails a concept you use a lot but don't have a word for, words just fall into that category of "noise that can be safely ignored". I know I have a lot of trouble remembering names of people unless I hear others use the name a lot, because the noise itself doesn't tell me much about the person in a useful way, nor is it often necessary unless I need to call them by name. Only when other people use the name and I need to connect it to the particular person to understand does it stick.

Anecdotally, I know I lost a lot of used vocabulary when I went to college, simply because the lexicon that got used at home was much broader than 18-22 year olds, and most professors, used. Likewise with jargon; stop being around people who use it and you rapidly forget, although it is easier to remember what it means than come up with the word yourself.

Expand full comment

I think the best metaphor for schools is playing team sports. Working out and practicing shooting or fielding obsessively is a necessary part of success. Structure and discipline are a huge part of the equation, but in the end, talent rules. Actual knowledge builds and fades like fitness and sports skills.

Expand full comment

I have taught students who were on Ritalin, went on Ritalin, went off Ritalin, etc. It doesn't do much long term for most students. It also dulls the emotions. It's a powerful drug. It's mostly "needed" because teachers and classrooms are boring and have weak classroom management.

Expand full comment

What happens if a kid who doesn't have ADHD takes Ritalin? Does it still help increase their grades and focus? It's widely reported that ADHD is overdiagnosed in kids, so you have to assume that many of them taking Ritalin don't actually have ADHD.

Expand full comment

I’m going to take this post as an excuse to link my favorite psychology lecture of all time.

Robert Bjork’s, “How we Learn vs How We Think We Learn.”

https://www.youtube.com/watch?v=oxZzoVp5jmI

It goes into a bit more detail on the Homer Simpson effect, but also the testing effect, interleaving effect, and spaced repetition.

An interesting take away: the intuitions people have about how well they learned material are usually wrong. Learning techniques that are objectively the most effective, are seen by students as less effective—even after using them successfully.

Expand full comment

People with ADHD can often focus quite well on things they are interested in. Of course school is standardized and boring as hell, so no surprise they have a hard time tuning in to the tedium.

Expand full comment

Interesting study but my intuition is that stimulants could still increase the amount that was learned by allowing the teacher to lead the class through more material in the same amount of time.

Teachers might do this fairly naturally if more kids, especially more disruptive or lower performers, were on stimulants?

What effect would that have on how much is learned?

What if teachers pace of instruction was varied experimentally?

(Full disclosure: I didn't read the study)

Expand full comment

As a person with ADHD, these results make sense to me. I feel like the challenges associated with it are greatly misunderstood. I can concentrate for short bursts, and my innate intelligence always carried me on tests. My impulsivity often got in the way and i would slip up when I didn’t read something carefully, but by and large I don’t think my ADHD effected my test taking too much. Where stimulants make a huge difference is in managing the day-to-day minutiae of life that for most people is second nature. For example, keeping your house clean, getting small chores done, paying bills on time, etc. To the extent that stimulants positively effect grades I expect it to be because students are showing up to class and turning their assignments in on time.

Expand full comment

In your 2017 post on adderall you cited a study where kids on stimulant medication, after 8 years, performed the same on all metrics as kids on CBT. I was a bit excited reading this post thinking "maybe it's just that school sucks," though it looks like the study also looked at factors beyond grades like arrests and psychiatric hospitalizations/

[1] https://slatestarcodex.com/2017/12/28/adderall-risks-much-more-than-you-wanted-to-know/

[2] https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3063150/

Expand full comment

"bottlenecked by some other resource"

There's a common observation made by nurses, I don't know if they have a name for it. Where a patient has long term disability [some blocked resource] later in life, they make a dramatic recovery, only to quickly die.

In the 80s, I trained as an EMT. I was required to spend a shift in the emergency room. I came back to volunteer often. EMTs brought in an elderly man, his wife said, in the previous weeks, he'd made a dramatic recovery from WWII head injury, and had been quite talkative up until that evening. Apparently, he'd been all but non-communicative for 35 years.

When we had a moment, the nurse explained this is a common occurrence, typically followed by death.

I find it plausible that some mental channel was blocked creating a backup, and this backup prevented other traffic. Perhaps some transient ischemic event either blocked, or released the blocking agent resulting in restored traffic.

Expand full comment

I've noticed Pelham etc. in a couple of the studies you've cited recently (or maybe I've been reading through archives and I'm conflating past and present). Are these quality researchers or something?

Expand full comment

I wonder how nicotine would fair.

Expand full comment

The idea of hitting the wall in learning based on personal limitations is experimentally accurate. I hit one when trying to study more complicated math than my brain would be able to cope with, despite being fine with the uni-level basics. As mentioned in previous posts, some people can't grasp hypotheticals and counter-factuals. Others can't learn to sing well. No growth mindset will help you beyond some personal limit. Once you hit it, learning progress stops being efficient (as in, polynomial in effort) and becomes exponential, or even super-exponential. There is a tiny bit of plasticity if one starts learning the skills early, but once you are an adult, you are hooped. Often you can compensate by getting at the same ideas from a different direction, where your brain has different limits, but simply scaling the mountain of knowledge that you see others walking up effortlessly is a fools errand.

Expand full comment

Personal experience—I've been taking Adderall for ADHD off and on recently. It does seem like it has an effect on my focus and motivation; I very rarely feel like just staying in bed and doing nothing when I've taken a recent Adderall dose. But, that extra motivation never seems to go into focusing more on work or prioritizing household chores more highly. Instead, I just spend that time highly focused on whatever new retro game hacking idea my brain spontaneously generated that day.

So I'd speculate that Adderall (and probably Ritalin as well?) is capable of giving people additional focus and motivation, but it's not going to make them *care* about tasks that they don't already care about. If mowing the lawn or paying attention to a school lecture isn't inherently interesting to you, then it's probably not going to become any more interesting to you after you start a focus-enhancing drug. And so you're not likely to prioritize it any more highly than you would otherwise, for so long as your focus-enhanced brain is capable of generating something more interesting to focus on.

Expand full comment

I started taking Adderall for ADD as an adult. I hadn't considered that I might have it previously because I got very good grades in school. During the process of getting diagnosed, I slowly realized how little I was paying attention in school. If a class was "boring" (which most of them were) I often brought a book with me and would read while the teacher lectured. If I didn't have a book. the teacher wouldn't let me read mine (most teachers left me alone because I didn't cause trouble and was smart, others were sticklers) then I would often read ahead in the textbook, doodle, or daydream. The only class I ever really took notes on were math classes, and only that because I couldn't memorize a formula just by listening to it. I usually wasn't paying close attention: if I tried to listen I often drifted off into other thoughts, would realize I hadn't been listening, listen for two minutes, and then drift off again.

So why did I get good grades? I dunno, I'm smart and I wanted good grades so I actually tried. I have good reading comprehension. I ended up something like 14th in my High School class, but I could have been in the top 5 if I wanted to be. Getting anything beyond an A average was just not interesting to me, I'd rather read my books.

I don't know where this ties in, but the over-under is that I think if I was on stimulants during High School I would have paid much better attention, but I doubt my grades would have gone up all that much.

Expand full comment

This articles comes to a comparable result - but by doing PET scans:

Methylphenidate Decreased the Amount of Glucose Needed by the Brain to Perform a Cognitive Task

https://www.ncbi.nlm.nih.gov/pmc/articles/PMC2291196/

> The biochemical mechanisms of action of MP have been well characterized: it increases extracellular levels of dopamine and norepinephrine by blocking the respective monoamine transporters[5]. It is unclear how these actions relate to its effects in attention and performance. Since dopamine and norepinephrine decrease background firing rates of neuronal cells increasing signal-to-noise ratio[6], [7], we hypothesized that in humans MP's dopaminergic and noradrenergic effects by decreasing non-task related activity should reduce the amount of glucose utilized by the brain while performing a cognitive task.

> There were no differences in money made (surrogate marker of performance) when the cognitive task was done with placebo ($47.60±4) versus when it was done with MP ($48.80±3).

My interpretation: All Ritalin does is *reducing* brain activity in non-task-related brain regions.

Expand full comment

I have a few things that come to mind.

1) ADHD symptoms correlate to trauma symptoms

2) The study is far from representative (173 children (77% male, 86% Hispanic, 10% African–American) with ADHD between the ages of 7 and 12 years (M = 9.2, SD = 1.4))

3) The study doesn't evaluate socioeconomic factors

4) The study doesn't evaluate language factors

5) The study has a highly male and minority sample, that is (generally speaking) more prone to socioeconomic concerns, linguistic barriers, undiagnosed trauma that affect academic performance

For reason 5, I am not surprised that the medication used had a negligible impact in this study. My personal and professional academic and mental health experience all suggests that ADHD medications can move the needle in some cases, but are (generally speaking) misprescribed, overused, and deliver little return. When they are used appropriately, they can have a transformative impact. I'm still waiting for a great study that quantifies the appropriateness of ADHD prescriptions.

Expand full comment

Let's not forget that a "summer camp" type situation is very different from a school. I would expect students to spend most of their mental energy on more social "stuff". Learning names, affinities, making friends, maybe establishing some kind of social ladder. Usually these things are simply more important to a teenager than learning vocabulary, but they tend to stabilize over a school year, and not over a month of attention training.

Expand full comment

> These students must be bottlenecked by some other resource.

I think the bottleneck for almost all school-related things student's interest in the task being measured. Most of the things kids are made to do in school don't have much bearing at all on the things they care about working on. Not-giving-a-shit can be a really strong component in the multi-factor equation leading to someone learning something (or not).

Most "good students" have had their motivation adequately hacked by teachers and parents who praise them for good performance, internalizing that extrinsic motivation. Most kids don't buy it. They'd much rather go build a fort, chat with their friends, etc.

Jumping through hoops to get gold stars is obviously a waste of time to many kids, so we have to find ways to allow them to pursue actually-interesting, actually-challenging, actually-useful work. Real, useful work creates opportunities for them to bolster reading/math skills to accomplish something that matters. It's the antidote to the not-giving-a-shit bottleneck.

For further exploration of this issue check out ASDE: https://www.self-directed.org/sde/

Expand full comment

These scores are incredibly low, especially given that the tests were multiple-choice questions with only four choices and that kids with IQ below 80 were excluded from the study. To get only 50% correct on the subject area on average, it seems like a significant fraction of the class was guessing no better than chance both before and after the intervention. (If 1/3 of the class got perfect scores, the other 2/3 did no better than chance in the post-test for the subject areas test.)

The study mentions that the test-givers read the questions aloud because they weren't confident the students were literate in English; but I don't think it mentioned anything they did to offset any lack of confidence that the students were verbally fluent in English.

Also note, the decision to test students by reading questions allowed (1) is a very unusual test-giving procedure, and (2) greatly amplifies the effects of concentration relative to learning in the results. And the combination of those two facts seems like evidence to me that that this experiment was trying to obtain the headline result it obtained. i.e. "medication doesn't help students learn."

Significant lack of fluency in English would also explain why the improvement on vocabulary was much better than the improvement in the subject areas (i.e. science and social studies). It's way easier to learn vocabulary in a language that you barely understand than it is to follow a history lesson in that language. (And is most of what you do in a learning-a-second-language class.)

The pre-test baseline of students scoring 40% on average on a multiple choice test of grade-appropriate vocabulary with only four choices also seems like pretty significant evidence that a sizeable percent of the students were not fluent in English. (That's what would happen if 20% of the class answered perfectly and the other 80% answered no better than chance, which is equivalent to saying that the class as a whole was able to recognize the correct answer [though not necessarily supply it] 20% of the time and randomly guessing the other 80% of the time! Recognizing the definition of only 1/5th of grade-appropriate vocabulary is not fluency when the grades in question are kindergarten through fourth grade.)

If you start with the assumption many of the students weren't fluent in English, "Maybe classroom instruction is redundant? That is, teachers say everything a few times, so if you miss it once, it doesn’t matter?" becomes a much more plausible explanation. For many of the kids it didn't matter because they weren't going to understand it no matter how many times they heard it. And for the rest, it didn't matter because they understood it the first time.

But my default interpretation of this study would be:

A third of the students were fluent in English. In the post-test, they knew all the answers to both the vocab tests and the subject area tests, so no impact of learning could be detected in them, only effects of concentration.

A third of the students knew enough English to understand English-to-English definitions. In the post-test, they knew all of the answers to the vocab tests, and guessed randomly on the subject tests, so no impact of learning could be detected in them, only effects of concentration during the vocab test.

A third of the students didn't know enough English to be able to learn English-to-English definitions, so they guessed randomly on all the questions, and the post-tests were not able to detect evidence of either concentration or learning in them.

No matter how real and significant an effect is, it is possible to design a study that is sufficiently bad that it can fail to find the effect. I suspect this is much more common than studies finding an effect that isn't real because it's much easier to do by accident. But I've rarely encountered an example full of nice round numbers that reeks of experimenters trying to hack a no-effect result as strongly as this one does to me.

Expand full comment

I think the prescriptions for ADHD to kids are more about "get them to sit down and be quiet in class" more than anything else. Some kids really have it very bad, with muscular spasms, facial tics, and involuntary noises so they do need something. But if it's "Johnny fidgets and talks and distracts the other kids in class", it's not so much about learning or even paying attention; I've daydreamed in class while sitting there quietly and looking as if I'm paying attention.

A lot of this is because teachers have to wrangle a class full of kids and it's easier to medicate the distracted ones into good behaviour. If they then do well on tests, that looks like "the medication is helping them learn". As everyone is pointing out, learning and paying attention is not the same thing as 'can finally focus and concentrate long enough to do a test'.

Expand full comment

It is too bad that the authors did not design the experiment (or at least analyze the data) with an interaction for IQ. It would be interesting to see whether it was possible to tease apart whether the medicine had a differential effect for smarter children, which would be a reasonable a priori hypothesis.

Expand full comment

Does learning plateau or are exams just bad at capturing learning beyond a certain point?

I don't know anything about surgery, but I'm reasonably confident that an experienced surgeon knows a lot more than an inexperienced surgeon. But the things he knows are difficult to test on a written exam, he probably knows things like "What to do if things start going wrong in this particular way", and "How to pick up the subtle signs that you might need to do X rather than Y in this procedure" and "Here's the exact amount of pressure and the correct angle that you need to cleanly cut this thing under these conditions". These things are incredibly important but won't make it into a textbook. Meanwhile he's probably slowly forgetting all sorts of stuff that was in his textbook but is irrelevant in practice.

Expand full comment

I've been a private tutor/classroom instructor for 12 years. My own experience suggests the following things:

- Ability to concentrate has a substantial effect on test performance regardless of the level of understanding of the material, so we should expect ADHD medication to improve test performance EVEN IF they did not learn more in a classroom experience

- I have taught "large" classes (20-30 students), small classes (3-5 students) and of course 1 on 1 tutoring. In my experience ~15-50% of students do not learn very much in classes larger than ~4-5. The issue in my experience is you need back and forth to make sure they're getting it. In a class of 4 there are still tons of opportunities for 1 on 1 attention. Not so in a class of 20-30. My hunch is that the classroom is having no effect for this reason.

Expand full comment

If Concerta helped (significantly) with test scores, then most college kids would be taking it. I doubt kids on Concerta are really paying attention more. Staring at the teacher, does not necessarily translate to paying attention.

Expand full comment

"These students must be bottlenecked by some other resource."

Yes, and it's something psychiatry alone can never solve: desire.

Expand full comment

Fascinating, but, the question not asked is why school in the first place?

From Monty Roberts, an internationally famous horse trainer and the first "Horse Whisperer,": As I have often said, there is no teaching, only learning. If you are a teacher, you have to create an environment in which your student can learn. You can't take knowledge and push it into a brain. The brain has to pull it in.

Expand full comment

I don't follow. How do we know Ritalin doesn't improve learning? In my classes I can only measure student performance by tests and assignments; if those are better, how do we know learning isn't? Do they give em another test while not on Ritalin? Considering it 4 my ADD kid, but not if it doesn't really help.

Expand full comment

Just as a personal experience addy doesn't really make a difference to me for HS style activities, even ones I'm bad at like writing. Those tasks just don't seem very attention limited.

OTOH trying to keep a math proof of an unsolved question in my mind is definitely helped.

Expand full comment

I have two sons that are eighteen months apart and rising 3rd and 2nd graders. They are both high functioning ASD and ADHD but present very differently. My older son has serious non compliance behavior issues and went on amphetamines early in his 2nd grade year. It really did nothing for him which wasn't a huge surprise to me because I truly see his behavior as substantially a maturity/emotional issue rather than impulsive disruption. He has no desire to appease adults and simply gets tired of things when they are no longer novel. We took him off the medication.

My younger son is definitely more of a "good boy" that wants to do well in school but cannot control his hyperactivity. He doesn't pay attention to the rules, etc. and then gets upset when he does something wrong. I actually think he would be a great candidate for amphetamine type medication as his inability to pay attention is really impacting what he gets out of school, sports, etc. My biggest reservation is really the philosophical objection to giving a seven year old boy drugs to make them more compliant for an institution and better participation in organized activities and balancing the reality that he needs help.

I realize this is mostly venting with little to the specific discussion but it seemed somewhat appropriate and this comment section is much better versed in this area than my own intuition.

Expand full comment

Interestingly, golf is one sport where this sort of ‘plateau effect’ is evident for all skill levels. A common comment from golfers of every skill level is ‘forgetting how to play’ and/or having to ‘relearn’ some aspect of the game. Greats like Jack Nicklaus and Bobby Jones to the worst hackers comment on this frustration. I expect that non-golfers just assume that once you master the game (or some aspect of the game) you’ll always ‘have it’, like riding a bike. But it doesn’t work that way. Could it be a brain limitation, where the various ‘swing thoughts’ can’t be properly categorized and then written into permanent memory?

Expand full comment

There is a developing tolerance problem with Ritalin and probably the three weeks results can't be extrapolated long-term.

Expand full comment

I think two things are going on here.

The first is approaching ability caps.

The closer you are to your ability cap, the harder it is to progress. As people approach their personal ability cap, they will have an increasingly more difficult time getting higher, resulting in progressively smaller gains. The better you are, the harder it becomes to progress even higher - something we see in a lot of things in technology as well. I suspect this is because you learn the easiest things, then have to work harder to learn increasingly harder things, and while it is not always the case that you learn all the easiest things first, it is common that you will master more easier things than harder things.

This is why the SAT has questions of different ability levels, and you'll find that the top students miss far more hard questions than they do easy ones, and some people may well miss nothing BUT the hardest ones.

I think, however, there is a secondary problem here, which is that we are measuring against a standard of perfection, when in reality ability levels don't really work that way.

Say we took some artists and had them draw stuff. They might screw up a few easier things but do things that are way harder than other artists do. These things might not be on the "test", but are more advanced skills.

As such, what we are really measuring with these things where we are measuring against a fixed, perfect data set may not be "overall skill level" but "error rate".

I have noticed that some people seem to innately have higher error rates than other people - they simply make more mistakes. I am not sure if this applies to every task, or just some limited subset of tasks, but no amount of repetition will ever cause them to be perfect at a task. While this seems like it might be related to intelligence, I'm not sure that it is; some intelligent people seem to have issues with this as well.

Everyone makes typos no matter how skilled a typist they are, for instance; I make far fewer typos than most amateur typists and type at a very, very high rate of speed, but I still make typos.

I have the suspicion that if you are measuring against some standard of perfection, at some point this innate error rate will dominate over all other factors.

I think it would be interesting to see if these medications cause people to "plateau" in their agility earlier, and/or if it lowers innate error rate on random other tasks.

Expand full comment

School is more than classrooms; it's a crucible for personal growth. Essays, reflections of our thoughts, refine communication skills. In school, we learn, socialize, and shape character. Essays nurture https://www.phdresearchproposal.org/ analytical thinking and discussion skills, crucial for development. Both school and essays are pivotal stages where not only knowledge but also essential life skills are cultivated. Together, they form a foundation for success in navigating life's challenges.

Expand full comment