Anecdata: I second the hypothesis that lowered standards (forgiveness and equity justified) were sticky post covid - but they were also accelerations of existing trends, not a sudden cliff out of the blue.
The once fashionable Education Reform movement (which thought George W. Bush's criticism of "the soft bigotry of low expectations" was the height of wisdom) favored by billionaires burned out in the mid 2010s as the impossibility of much narrowing the racial gap in school performance sank in, and was replaced by wokeness and its emphasis on how racial gaps are due to white evilness and that whites therefore must pay. Not surprisingly, wokeness discovered a way to lower the racial gap in flunking students: don't flunk any students.
Out of the loop - why was he canceled? Because discipline consequences were naturally going to fall on poorer and minorities more and so were inherently racist or something?
Yes, but is it still the same COVID? My impression was that the wildtype and beta variants were particularly nasty, unusal pathogens, wreaking havoc in unexpected places, while the variants that replaced them were, while ridiculously infectious, much more standard respiratory viruses with effects more in line with a regular cold or flu.
That was my impression as of Christmas 2021: suddenly, everybody has come down with this new Omicron variant, but, remarkably, people aren't dying in the numbers you'd expect.
Right, for example the Delta variant from India in mid 2021 was nasty, but the Omicron variant that arrived in America in late 2021 was highly infectious, but not as lethal.
My impression is that basically we got lucky in that covid evolved in South Africa in later 2021 to be less ferocious but more infectious, so by early 2022 after Omicron faded, we were largely over the hump with covid: most people had by now either already been infected or vaccinated or both, and the subsequent variants weren't as bad as pre-Omicron ones.
But I don't hear many authorities saying that anymore when it comes to this fifth anniversary.
Maybe there is some deterministic reason why it had to evolve that way, but diseases don't always: e.g., smallpox stayed vicious for thousands of years.
This seems unlikely to me, but probably worth examining to be sure.
Quick googling suggests that surveys find something like 6-8% of adults report themself as ever having had Long Covid. (A smaller fraction currently have it, but this many report a period of several months where it affected them.) I think that numbers are generally lower in children, but it's not completely out of the question if it's higher.
If 6% of students had Long Covid, and this caused just a minor dip in their test scores, this probably couldn't explain such big overall test score dips, so it would have to cause big drops for the students who were affected, if it's going to cause this.
That would mean that most of the students with Long Covid would probably show up in the "lower performing students" group, and not the "higher performing students" group - the "higher performing students" would have their test scores drop a little bit, as 6% of their number moved out and are replaced by the top of the "average students" but the "lower performing students" group would probably drop more significantly, as 6% of their number moved down, and many of the rest of them are replaced by people from higher groups that had dropped to below the old "lower performing students" group.
That *might* be compatible with some of the data, but it seems unlikely to me.
I don't think it's plausible that 25%+ of young children have long COVID, unless we're completely misunderstanding the situation. Also, if that were it, why wouldn't high-performers be affected?
It doesn't have to be the kind of full on long covid that people report, which stops you working or at least gives you noticeable symptoms. A small amount of circulatory and/or brain damage would presumably have similar results in knocking the performance of marginal cases down a bit, and wouldn't necessarily be noticeable?
A _lot_ of my friends report reduced energy levels that never quite fully recover after their first tangle with covid, not to the level they'd go to the doctor about it, but noticeable for those who pay attention.
I find it extremely plausible that lots of 80 year olds came down with severe covid, eventually recovered, but found that their cognitive abilities had taken a sharper downward turn than simple aging would cause. Getting old can be like the Hemingway character who went broke in two ways: gradually, and then all at once. A major illness can cause a sizable downturn that you never recover from. For example, if you look at pictures of my mother before and after her cancer at age 61, right before she looks middle-aged and right after she looks elderly, and she stayed elderly.
In contrast, I find it less plausible that lots of grade school kids were damaged long term by covid. Maybe, but kids tend to pop back from all sorts of vicious ailments.
There are recent studies showing that covid can subtly decrease cognitive function without patients even noticing. There are tons of studies that show covid can cause changes in the brain in even mild cases. I do not think that self described rates of Long Covid are accurate.
These are just two studies out of dozens in this vein. I am not a scientist or researcher, but I do think that they're on to something based on my anecdotal experiences.
Judging by self-report, long covid seems like an ailment that afflicts the better educated, such as New York Times subscribers, more. But the biggest drop in NAEP/TIMSS scores has been among the lower half of society, the free lunch kids.
The better educated are more able to advocate for themselves and will therefore identify unclear symptoms as long covid rather than just feeling a bit crap. Meanwhile small neurological defects will affect people who didn't have as much margin to deal with problems more.
What? Of course dead people don't have subtle long-term effects. But those who had a mild form of the illness, even one that went unnoticed, certainly can. We know, for example, that adults 50+ who experienced a mild case have significantly increased risk of heart attack. There may well be other effects that we don't yet know about.
The "wildly speculative" hypothesis that there may be long term effects that aren't correlated with the observed severity of the illness? I gave one example, the significant increase in the risk of heart attacks. I don't know that there are others, but we don't know that there aren't.
I meant more generally the notion that some sizeable portion of children have long Covid. AFAIK the thing that correlates most highly with long Covid is trait neuroticism.
Long COVID is a specific-ish thing, and I'm not saying I believe lots of kids have it. I'm saying it's possible there are as-yet unidentified long-term effects.
Well, one might start looking at this by looking at other countries with same Covid levels but different policies. Would be useful as a comparison to US NAEP scores, as well.
Wild speculation, but maybe it affected teaching staff especially, and this could partially explain the lower scores? "1 in 5 educators who, according to a recent EdWeek survey, have experienced the emerging, mysterious illness known as long COVID." in this article https://www.edweek.org/leadership/1-in-5-educators-say-theyve-experienced-long-covid/2022/04. This 2023 paper from the UK (https://pmc.ncbi.nlm.nih.gov/articles/PMC7615205/) seems to say (I'm not sure I understand the "likelihood (ORs)" stuff) that people in teaching have 11.6% higher chance of getting long covid, and that 22% of people with long covid in teaching report a reduced function by "a lot". The raw prevalence of long covid symptoms in "teaching and education" slowly rose from ~2.5/3% up to ~5% (Figure 2). I'm not sure what's the impact of ~1% of the people working in education having a reduced function by a lot (22% of 5%, roughly).
Sequela from viruses is not a fake or made up condition. Some people get it from the flu, and there are reports of people getting it from the original SARS outbreak in the 2000s. There is a possibility that some people are making it up, but long covid is well studied enough at this point that to dismiss it as psychological is ignorant.
There are quite a few people who barely survived covid -- e.g., I can recall reading an article about an ER surgeon who spent a month in intensive care on one of those heart-lung machines reserved for the most desperate cases. Amazingly, he lived (ER surgeons tend to normally enjoy rude good health), but I doubt if he'll ever be quite the same vigorous man he once was. If that guy said he was a victim of long covid because he could no longer handle working in ER, I'd say, "Yes, sir."
But long covid enthusiasts tend to argue _against_ there being a correlation between how severe their case was and how severe their subsequent debility is.
Perhaps they are on to something. Or perhaps they are hypochondriacs. Or perhaps some are the former and some are the latter.
That would require evidence and a plausible epidemiology. Long Covid is rare, especially among children. It would also show up in IQ tests and AFAIK there hasn't been anything unusual happening there. SAT scores actually went up slightly post-Covid.
Teachers I know got burned out/fed up and quit [during and after covid]. I haven't seen any data to know if that is a widespread phenomenon, but if you're looking for why schools could be worse now, that's where I'd start looking.
This was my thought too, as a current teacher. The profession is much easier to get into now than it was 10 years ago, due to widespread teacher burnout. The loss of experienced math and English teachers could have a big role to play (and is potentially testable).
Mostly, we'll miss the part where you were earning and investing money, rather than cashing it out to finance retirement. And really, each of you is going to miss the part where all the rest of you were earning and investing, when you each try to retire and cash out.
I'm not finding a lot of data to corroborate either (apparently teacher turnover went from 20% / year to 30% / year, with high poverty districts affected to an even greater degree), but my money is on teachers as well. I bet a lot of teachers changed district (eg relocating closer to family), changed methods (eg remote teaching), or left the profession altogether during COVID, including very experienced, top performing ones, and the void they left hasn't been filled yet, because the new blood hasn't ramped up and acquiring teaching experience is slow. Simple data like "average years of teaching" for 2014-2024 could answer that, but I'm not finding it.
>I'm not finding a lot of data to corroborate either (apparently teacher turnover went from 20% / year to 30% / year,
Going to 20% to 30% seems huge to me!
First of all, it my only be a 10% change in absolute numbers, but it's a 50% increase! Most systems cannot change a sudden 50% increase in volume of some major fundamental challenge they face.
But more importantly, the fact that it is a yearly rate means it is cumulative, having exponential effects on seniority.
If we model the 20/30% chance/year as hitting teachers at random (which is wrong of course, but illustrates the point), then at 20% you get 32% of teachers making it to 5 years, and 11% making it to 10 years. At 30%, you get 17% making it to 5 years, and 3% making it to 10 years!
So it feels like that 10% absolute change would be plenty to massively decrease seniority among teachers, which we could reasonably expect to have a big effect on education quality.
It does depend on where the dropout is concentrated. If you had a lump of retirement, and then any new teachers drop out much more often, you could get the larger portion of teachers who have been there for a while staying there, plus a churn of new ones.
New teachers haven't got their lesson plans set up yet, etc.
Why would this be a new phenomenon? Have teaching circumstances changed in the last few years?
Teachers tend to be white because there isn't much affirmative action in hiring public school teachers due to a 1990s federal law promoting good teaching. On the other hand, there is huge affirmative action among principals and administrators. Hence, there has been a quiet racial cold war going on in the public schools between the teachers and the administrators.
So, during the Great Awokening, especially during the Racial Reckoning that began May 25, 2020, it hasn't been unusual for white teachers to retire. Stocks are high, so retirement accounts aren't paltry. But are retiring teachers recommending to their kids that they go into teaching?
This is also my standing hypothesis. No data (and no faith in teacher quality rankings anyway), but lots of teachers in my extended family and they all say there loss of many of the best teachers has been intense post-covid.
Even more subjective (though it should be possible to study) but I think covid coincided with (and partially drove) a new level of politization of public views of schooling, more intense if unhelpful fights about curriculum, etc. I would expect all of those things to degrade schools over time.
Anecdotally it's the same here; those close to retirement are taking it earlier and those not close are fleeing the public schools, either for non-teacher jobs or to teach at private schools.
Stated reasons all seem to have a theme of reduced autonomy in the job (the two biggest reasons I have heard are more stringent curriculum requirements and inability to discipline problem students, but there were other minor complaints that my brain filed as "administration trying to control teachers").
I know I saw a review article in the last year that put the figure at 5-10% but I can't seem to find it. This is what ChatGPT said:
Chetty, Friedman, & Rockoff (2014) – Their study found that high-quality teachers significantly impact long-term outcomes, such as college attendance and lifetime earnings. However, teacher quality explained about 10-15% of the variance in short-term student test scores.
And frankly I'm skeptical of 10-15. Generally I would expect this topic to be a nexus for motivated reasoning. The political incentives lean strongly towards valorizing teachers and minimizing the role of genetics, so I would expect the published literature to overestimate actual teacher value.
It's easier in that you can pretend everyone is out there listening and taking notes and thinking about what you are saying. But they're not, and when test score come back, you go WTF!!! How am I going to get them engaged?!?
But the will to fix it might involve taking away power from the people who don't want to fix it, which is further motivation for them not to measure it.
We spend a lot of money and attention on this topic as a society, and most politicians at least talk about their plans for it.
Look at teh first graph in this post, before the current drop-off these scores were steadily increasing for decades. We absolutely *do* do things about this in general, we're just facing a current acute problem that people are trying to figure out.
And it is impossible to fix this problem as long as there is a teacher's union to protect incompetence and make sure bad, ignorant teachers are never, can never be fired.
My entire career originated with Congress allocating in the famous 1964 Civil Rights Act the sum of one ... million ... dollars to James S. Coleman to demonstrate quantitatively that the reason for the racial gap in test scores was due to less spending on black than white students.
When looking at education statistics one needs to keep a few things in mind:
a) The population changes over time. E.g. California was ~87% white in 1970 or so. Today California public school classrooms are over 50% Hispanic. One can hope/pretend/assume that those two populations score equivalently ... or not.
b) The population changes over time. How has private and homeschooling percentages changed over time. If more of the top scoring kids aren't being tested then we'd expect the test scores to drop.
c) The tests sometimes change where they put cut lines.
Yeah, but look at the graphs. It's not a gradual change since 1970. It's going up until 2015, going slightly down until COVID, then (on at least some of the tests) plummeting over COVID, then going a bit back up later. Percent white doesn't look like that at all!
A (medical) doctor named John Jacob Cannell looked into state education test scores in the 1980s and discovered that all 50 states were above average. He spent some time investigating and wrote a short booklet title "The 'Lake Wobegon' Report: How Public Educators Cheat on Standardized Achievement Tests."
It generated a tiny bit of news coverage but no change.
My working assumption for most/all US public education testing data is that it is badl flawed. On purpose.
On top of that we also have sampling problems and scoring problems. But the raw data should be considered suspect (in either direction) until proven otherwise.
Do we think the data became significantly more or less cooked in the last few years? If not, I'm not sure how much explanatory power it has for the current topic. But I'm looking forward to reading the article at the link - thanks for sharing.
If capitation grants depend on results, there's a strong incentive there to cook the data. I don't know if that's so, but it seems plausible. Nobody wants to attend or to fund School A where the results seem bad (because they are poor, because the returns are honest) so it's in the interest of School A to nudge the worst performers out the door (or at least prevent them from taking the tests) and fudge the remainder to improve the appearance of things, simply for survival.
The NAEP results don't look terribly faked. They mostly look like what most of the other data validates.
The most dubious NAEP bias strikes me as Texas's somewhat high scores. Texas appears to take efforts to encourage it's dimmest 5% of students to call in sick on NAEP testing day. But it's pretty marginal.
There is currently a "teacher test cheating ring" scandal in which a not-small number of teachers paid someone to take the test that allows you to become a teacher in Texas. The locus was Houston but every few weeks uncovers a few more teachers in distant parts of the state, who paid someone to take the test for them - up to about 500 now. I don't know if this is a hard or an easy test. But either way, it suggests a culture unlikely to regard testing integrity as a goal.
My basic point is/was that the data is unreliable. In both directions.
Partially because the folks reporting it collectively don't want it to be reliable.
I *could* dig around and try to figure out what is going on. But that is work and feels a lot like forensic accounting where you try to figure out where the losses have been buried.
I don't find that interesting here so I don't feel a need to explain the results. Since I don't trust the data they don't matter.
NOTE: I got to play this game about 15 years ago in a non-education field when GPUs were *just* starting to become popular for high performance computing. Lots of papers got written with 20x, 50x, 100x speedups over CPUs [the record was 3,600x speedup over a CPU solution]. When my group ported some code we found ... 4x at best.
What was going on was that there were a number of ways to sandbag the baseline and different groups used different mixes of those ways to determine how much faster their GPU code was than their CPU code. They made their CPU code slow to show a great speedup.
I'll further note that the claims were coming from university research groups and being presented at conferences. This wasn't marketing nonsense from Nvida.
When I dug into this 20+ years ago for US education statistics I found similar games (though different strategies). The SAT was made more coachable a number of years ago, for example. And the SAT taking population changes based on things like states requiring students to take it to graduate.
Education achievement statistics are full of this sort of nonsense.
For the GPUs I no longer feel any need to explain weird results. Either the folks reporting the results explain what they did and why they got the result and how it makes sense or they don't. But I'm not interested in trying to reverse engineer.
Finally, 15 years ago when I eventually got an Nvidia support engineer on the phone to discuss my 4x we walked through what was going on and he said, "Yeah, 4x is about right." The other groups *were* scamming the numbers. And Nvidia knew it as well as the HPC research community as a whole. No one cared.
It's tiresome for people to assume that nobody has ever looked into test scores before, so therefore they must be obviously "cooked."
The most notorious thumb on the scale of NAEP scores is that you have to have 90% attendance on the day of the test, and Texas tends to have 91% attendance, while other states often have 95 or 96% attendance. It looks like Texas's high NAEP scores relative to its demographics (e.g., compared to ethnically similar California) is in part due to Texas encouraging its 5% dimmest students to take a day off when the NAEP comes to town. But, that's also not a really huge deal.
In general, the NAEP is administered by professionals who want to do a good, fair job.
This is a high-value comment. Thanks for this info, it's super interesting. Kinda makes you wonder how much of the country is caught up in pantomimes like this in order to fool the people who control the money. I suspect some large fraction of government- or charity-sourced funding is being directed by shell games like this.
It just illustrates why maintaining a profit motive in all things is so important. Everything immediately becomes dishonest in its absence.
I agree that point a) makes little sense (or perhaps means that more Hispanic students has contributed to a steady increase in NAEP scores in California until ~2017?), but point b) might have merit: during COVID, tons of families who could afford it moved their kids to private schools (not because they were good schools, but because they were open), and these kids aren't taking NAEP scores. If these kids used to pull the average up (and it's plausible that parental income and IQ are correlated), that mix shift might explain the drop.
No. I’m glad you mentioned it. We should teach basic statistics in secondary school (that’s high school to you) even for kids with little math ability just to be aware of statistical tricks. Zooming in is one of them.
There’s actually very little change in that graph. It’s largely steady
I always hear 'we should teach [important thing X] in school'. As if people remember or retained much of all the other thing and skills they are supposedly taught in school already.
Illegal immigration massively spiked in 2021 with the Biden administration, which would look on a time series similar to the aftermath of covid. Given the younger demographic profile of migrants (whose kids might be in school even if the parents are undocumented) and the shrinking pool of native born children, adding millions of young families in the span of just a couple years could have an outsize effect, especially when you consider that (apart from their own test scores) an influx of non-native speakers likely has disruptive effects on everyone else's learning. And IIRC, the most common finding on school quality is that ongoing class disruption can bring everyone's learning down much more efficiently than widespread good behavior can bring anyone's learning up.
"a) The population changes over time. E.g. California was ~87% white in 1970 or so. Today California public school classrooms are over 50% Hispanic. One can hope/pretend/assume that those two populations score equivalently ... or not."
Unlikely this has anything to do with a decline over 6-10 years.
"How has private and homeschooling percentages changed over time. If more of the top scoring kids aren't being tested then we'd expect the test scores to drop."
They probably didn't get ignificantly more private schools running over the last 6 years that siphoned off good performers, so more homeschooling taking the best performers out of circulation would be the only real explanation offered here.
Statistically, homeschool did massively spike, and it was almost all the parents well-positioned to help their kid, start a "pod", etc. Anecdotally, I have several friends who took their high-performing kids to homeschool during covid and just never went back.
I know of two groups of parents who started a "pod" with a professional teacher, and they kept it going post-lockdown because everybody involved preferred the situation and could afford it. They both stopped when the kids were secondary school age.
Also, the local Catholic schools can be a good bell-weather for parents' perception of the public schools, since they are typically the most affordable private option. E.g. when our local school district eliminated honors classes, the enrollments of non-Christians went up, and there was a similar bump when they reopened sooner than public schools.
In my state the "standardized" state test changed three times in 5 years. Makes it really hard to compare data or know what is actually going on. Not that it matters. Bad schools seldom face consequences that can make any difference for the kids.
The post mentions a couple times that in 2026 when students who started after covid reach 4th grade and start taking these tests it'll provide more information one way or another.
I would not expect a sudden reversion to pre-COVID scores - I have a child in that cohort and their first-grade teacher described that year as "half the class is basically feral, and I'm doing triage to get everyone to grade level." That will show up even in the high-performance kids, because they're not getting the attention they would normally get in a class because they're doing well enough. I think you won't see kids truly unaffected by COVID until the 2028 or 2030 numbers.
As for the pre-COVID drops, I'm surprised phones weren't mentioned as a culprit. Would explain the larger drop in the 8th grade cohort vs. the 4th.
Like with a lot of stuff, I think Covid accelerated trends that were already happening. It seems pretty clear a big chunk of the population really doesn’t think of school as a priority and wants teachers to take on 100% of the responsibility for teaching their kids, with zero help from parents. I also suspect that fentanyl (and other drugs) took a decent number of parents and caregivers out of the picture. And of course screens are a huge part of it.
I think the analysis about missing a year of school assumes a reasonably intelligent kid with parents who are involved . I suspect the impact is much, much greater with a kid on the left side of the bell curve who doesn’t seem the inside of a school for a year and loses the habit of attending, along with most of his friends.
> Like with a lot of stuff, I think Covid accelerated trends that were already happening.
Yes, exactly. Test scores were already in decline due to systemic and/or cultural problems; COVID just time-warped the trend forward by about a decade; and my prediction is that generative AI will warp us forward just as quickly (if not faster):
I think you’re overestimating the prevalence. It was surprisingly hard to find data, since almost every source only trumpets the overdose rate. But I’m seeing estimates below 0.5%. Is that really enough to compete with more common factors like “remote learning” or “every teacher who can afford it quitting”?
Teachers I know say that behavior and mental health (already declining) eroded markedly and never returned to pre-Covid levels, creating challenging learning environments. That has pushed teachers to leave the profession. 50% leave within 5 years of starting and veterans are seeking early retirement like never before.
I've heard the same from multiple teachers. Behavior issues got significantly worse and disruptive students are frequently causing issues leading to "room clear"s.
They complain that administration is just not helping much and teachers are directly having to deal with extreme and violent behaviors.
The argument I hear, but haven't researched, is that teacher shortages might be to blame, caused by poor pay and poor treatment (by admin and by students; I wouldn't be surprised if political and cultural polarization is causing kids to be more chaotic and obstinate).
Anecdotally, I live in a pretty good school district in a very good school state, and teacher turnover has been awful. My kid's school barely has enough teachers to function, lacks needed specialists, and is facing a significant budget shortfall next year. A high school in a nearby town has some classes that don't have teachers; the kids just go to the room and...hang out, I guess? I suppose the diligent ones probably work on homework. But this isn't a poor town, they just can't find teachers.
Has teacher pay gotten substantially worse? Have staff shortages? (My gut sense is that the answers are “no” and “yes”)
Teacher pay has been blamed for every negative metric for decades, which is not to take a position on whether their pay is sufficient, merely to say that I don’t think that is something that changed in 2020.
I think teacher pay hasn't gotten absolutely worse, but pay in other jobs has gotten significantly better since the pandemic (there were a couple years of significant labor shortages).
Wage inflation is almost exactly what Baumol's cost disease is.
Specifically wage inflation in sectors that haven't seen a productivity increase, as a sort-of spill-over from sectors where productivity has increased.
Baumol's cost disease is specifically about productivity enhancement in other fields. I have not seen anyone claiming that the pandemic caused a bunch of fields to experience productivity enhancement. Pandemic (and post-pandemic) wage inflation is generally attributed to retirements and employment reshuffling where employers had to offer increased wages to fill vacancies. One could certainly argue that Baumol's contributed to changes in the quality of teachers over the last 30 years but that is somewhat removed from the comment to which you were responding, "...teacher pay hasn't gotten absolutely worse, but... since the pandemic."
That would be my suspicion as well, though the question of pay is interesting, as teacher pay as the unions fight for it (i.e. more pay and greater job security) probably doesn't help much, but if the best teachers truly are leaving for less stressful jobs with better pay, we want some way to get them back. If it's not pay it needs to be about stress/chaos. Is that an administration problem or a broader cultural one? Probably a bit of both, but either way it's unlikely to be an easy fix.
So the most elegant solution might be to tie higher pay to performance. But how do you measure performance? It's not as simple as many other jobs. It can't be raw test scores as that would just funnel the best teachers to the richest school districts. Test score improvements could work, but I imagine there are diminishing returns over time as scores enter and move past the fat part of the bell curve. Reputation? Peer accolades? That could incentivize inter-departmental conflict in a field that should be extremely collaborative. Reviews from students/teachers? Could pressure teachers to never fail anyone.
I don't think there's an easy answer. I suspect it comes down to the messy cultural stuff. But if a core issue is that many of the best teachers are leaving, some way of bringing them back might help push back against decline. People who have actually researched this will know more, I'm sure.
The most elegant solution is to let private organisation figure out how to attract good teacher, or otherwise how to provide good education for their customers.
Just like we let private organisation figure out how to attract good staff to server their customers in most other parts of society.
(I'm deliberately not expressing an opinion over whether this is a _good_ idea. Only that it would obviously be the most elegant solution, by virtue of removing a special case.)
If the pay is the same but the job is worse (misbehaving children, misbehaving parents, misbehaving administrators...), then it follows that people would quit.
I think that the work / pay ratio has become significantly skewed. Teachers are expected to enforce tons of policies meant to rectify society and family failures or weaknesses, like mental health programming, behavioral programming, free menstrual products, addressing drug use in the classroom, being a mentor to kids without good parents, etc., as well as tons of special interests, like mainstreaming delayed children. They work every night and every weekend, for pay that isn't keeping up with the increased workload.
If it's the teachers, that could have been exacerbated by Covid. I have a friend or two in the school system and they were pretty unhappy about how their administration handled the pandemic, enough that from what I heard lots of people who could retire or swap jobs did.
From my (mostly high school) perspective, chronic absenteeism looks likely to be a big cause. Administration has been cracking down heavily this year, in a way that makes me think it’s been much worse than usual in the last 3-4 (this also matches anecdotes from veteran teachers).
[My school is kind of a weird one though (“public magnet”, ish), so take these ideas with grains of salt.]
We’re also attached to a more conventional middle school (a fairly poor-performing one in a fairly poor area). And I hear stories from teachers who spend more time with the middle schoolers. There’s one teacher’s experience in particular I want to relate:
Mr. H was a literature teacher for a few years around a decade ago. Seven years ago, he left to do a PhD, and then he came back last year (as Dr. H). He had both middle and high school classes, and related to me a number of stories of total disarray resulting from poor standards and insufficient administrator attention.
Dr. H was particularly concerned with attendance among the high schoolers—he expressed in no unclear terms that absenteeism had become much more common since his previous tenure. And he told me that the middle schoolers were total agents of chaos—fighting in the halls, cursing administrators and teachers out, and facing almost no consequences. One or two day suspensions at most. He made it through a year and a half, then earlier this year, Dr. H left our school.
This is all anecdotal, but is pointing to two main issues from my view (and I think some other commenters are saying similar things): lower standards (academic, attendance, and, I think crucially, behavioral), and good teachers leaving. Even those that weren’t burnt out by Covid directly (like Dr. H) were driven away by the total structural failures that it created.
It is not possible to handle large groups of people, if there is nothing you can do about those who choose to be disruptive -- because, statistically, there will always be someone like that.
On internet, you have bans. In real life, you have the police; people who break the rules can get fined, or put in jail. Voluntary associations can kick out misbehaving members.
At school, in the past we had corporal punishment, which is no longer legal... and that is good in general... but the problem is that we have no replacement. Or rather, all suggested replacements have some serious problem. The results are... predictable. (Although apparently the people who make the rules are unable to predict even the most obvious things.)
The replacement is kicking the student out of class every time they begin to misbehave. Chinese schools no longer have corporal punishment either, but misbehaving students simply don't exist in Chinese classrooms because they know there's zero tolerance for that kind of behavior from the teacher, the principal, the parents, the classmates, or anyone else.
But it requires support from the school. As a teacher you can't simply send the misbehaving student away (they would probably misbehave in the hall, and potentially hurt themselves, and then their parents would sue you). You need someone you can call to take the student away, and some room where those students are collected. So this must be official school policy.
And even then, I am not sure whether it is legally okay (this depends on specific country's laws) to kick a student out of too many classes. At some moment, the parents could argue that the student is deprived of mandatory education. So you need the entire system to say that yes, this is perfectly legal.
So while the solution exists in theory (and in China in practice), no teacher can implement it alone, and maybe not even a single school.
I don't think anyone really has a satisfying explanation of what's going on. Even weirder is there are assessments of adult skills that show worse performance over the same period of time. (https://www.wsj.com/us-news/america-us-math-proficiency-falling-1b5ac73c) During this period there's been a shift to more online assessment, but test-makers typically check for that sort of platform influence so it doesn't seem like a legit explanation.
Did we all become dumber due to catching Covid? What happened with all those studies that showed long term damage to grey matter, etc? Unlikely to be the explanation, but still somewhat relevant
The decline started with the post-trurth era. Nothing matters, only soundbites from short vertical videos matter!
It's absolutely ridiculous how many commenters here seems to have the obvious psychocultural shift in their blindspot. (Scott included.)
Kids also are famously not living under a rock, they see what's going on in the world, see AI is coming, wars starting, climate changing, and ... who the fuck wants to study for tests nowadays!?
This is a theory. Ideally it would be useful to check whether it is supported by data. Tik Tok user base did in fact grow by a factor 10 between 2018 and 2022, so the timing seems right. This is a starting point but it's a far cry from proving a causal connection
I can't see the article. What specific test are they talking about?
I wonder if short-form video has contributed to this. I imagine someone has already analyzed this sort of data stratifying by short-form video consumption. That would make sense, since apparently this worsening hasn't happened among high performers, who we could suppose might consume less short-form video.
This, my theory is that smartphone and social media addiction are the primary drivers of decline, and COVID resulted in an acceleration of that trend due to everything going online and people being forced to stay inside all day. Even after everything reopened the increased tech dependency remained. You can also see the same trends appearing in assessments of youth mental health, which itself also affects educational ability.
Autism is diagnosable if it leads to adverse effects. Likely more people than assumed has the genes for Autism, but I would argue that many of those people find coping mechanisms and never get diagnosed. I would say that technology becomes a special interest for many folks, neurotypical and diverse, but imo technology accelerates neurodivergent people towards Autism.
It has little, if anything to do with covid. As you observed, the decline started before the pandemic.
The real reasons in my opinion:
1. Educators for years have been shifting to teaching strategies that don't work (e.g. "whole language" learning, "inquiry-based" or "number sense" math, virtual elimination of science and social studies for elementary and middle school) because traditional methods are boring for teachers. My girlfriend is an elementary school teaching assistant. Her school switched back to teaching phonics this year and younger kids reading scores are skyrocketing. The kids beyond about 3rd grade are still struggling, probably because they missed a developmental window for learning to read. It's sad.
2. Elimination of physical textbooks. My kid is expected to reference a text and write a response on an 8" tablet. After accounting for the virtual keyboard the text and the word document are relegated to two tiny ~2x3" squares.
3. Increased absenteeism is partly becuase a) Everyone is sick all the time now and b) Because during those years of "virtual learning" parents were finally able to observe how terrible and inefficient modern classrooms are. Once you realize your kid's school is bullshit, your drive to make them attend school is diminished.
Re: 3, I think mores changed a lot here. When I was a kid, you went to school unless you had fever or were vomiting, staying home because of anything less than that was malingering. That clearly changed a lot during COVID, I don't think it totally returned to what it was.
Or at least, if you insist on technology, buy large screens.
The current situation is worse than both extremes: we want technology because it is cool, but we buy crappy technology because a good one is more expensive.
Writing on screens is a bad idea. I could imagine reading from screens, but they would need to have the same size as the paper textbooks. I checked some e-shops, the readers of that size cost about thousand €. (I imagine that this is partially because few people buy them -- if the educational system made a huge order of a few hundred thousand pieces, that could probably drive the prices down.)
Rereading my comment, I kinda conflated 2 things above. This is what I get for writing fast and on my phone. For item 1: teachers abandoned phonics and rote math instruction because it's boring and because of modern teaching fads. Science and history/social studies are deemphasized in elementary and middle school education to make more time to teach to standardized tests and because of some weird theory that young children don't absorb concrete facts well. To the extent you do see science or social studies taught in younger grades it tends to be vague stuff like "what is a community?" rather than explicit facts.
I also think science hard or soft are sacrificed because we need to focus on fundamentals because the faddish methods failed, so obviously we need to try them harder and longer, right?
> some weird theory that young children don't absorb concrete facts well
This goes completely against everything I know about how people learn. Concrete facts are easy; abstractions are difficult if you don't know the concrete facts.
But yeah, the fads in education usually go both against the science and the common sense.
Maybe the decline partly owes to growth in the population of students who are illegal immigrants. They don't speak English and received poor educations in their home countries before coming here. Average them into the standardized test scores and they drag it down.
This doesn't seem particularly likely as an explanation, seeing that this effect seems to be pretty similar in nearly all states, while immigration (both legal and illegal) is concentrated in some states and nearly non-existent in others. West Virginia, Montana, Mississippi, South Dakota, and Maine are the five states where less than 5% of schoolchildren are from immigrant households, and they are all over the chart with some having the worst declines in recent years while others having relatively low declines.
Similarly, the six states where children from immigrant households make up more than 30% of schoolchildren are California, New York, New Jersey, Florida, Texas, and Nevada. These states are all right in the middle of the pack.
If you think that the distribution of immigrant children in the past few years is different enough from the distribution in 2021 to cause some significant errors in this comparison, I'd be glad to learn of some relevant data.
Do the numbers for this match up in *any* sort of a remotely plausible fashion? Have you checked?
My recollection is that numbers of undocumented immigrants in the U.S. has remained pretty steady for the past decade and change, in the range of 12 million or so[1]. Not only is it missing the sort of sharp increase that would be necessary for your argument to hold, but it's just plain to small anyway: a demographic that makes up 3% of the total U.S. population isn't going to have such a dramatic impact on test scores. Not, it's possible that the proportion of school aged kids within the demographic is larger than the percentage of the population, and has grown relatively larger in recent years. But we'd be talking about a BIG demographic difference. Where's the evidence for that?
[1] Though of course "total number remains steady" is not the same as "all the same people are in the group." Some die or leave, others arrive.
"numbers of undocumented immigrants in the U.S. has remained pretty steady for the past decade and change, in the range of 12 million or so[1]."
Since we know that large numbers are crossing the border into the US every year, this would require equally large numbers leaving every year. This is not happening. So the number is not true.
It's especially not true since that *same* number (actually "11 million") was being quoted back during the Reagan amnesty thing forty years ago.
Yes, I'm old enough that I was around and interested then. I do remember things. No, it was not believable then and it is definitely not believable now.
"My recollection is that numbers of undocumented immigrants in the U.S. has remained pretty steady for the past decade and change, in the range of 12 million or so"
And an increasing share of those illegal immigrants are now minors: "Beginning about ten years ago, the pattern changed and unaccompanied minors and migrant families from northern Central America accounted for a pronounced share. This trend has sharply increased since 2021, as many irregularly arriving migrants have come from elsewhere in the Western Hemisphere and farther afield, often seeking asylum in the United States (see Figure 2). "
"a demographic that makes up 3% of the total U.S. population"
The share could be significantly higher than that. No one knows for sure.
"The share could be significantly higher than that. No one knows for sure. "
And yet, it *still doesn't matter.* It could be 4%, 5%, 6% or even 8%--hugely more than the number I stated--and it still wouldn't come REMOTELY close to carrying the weight of the above argument. Look at the graphs again. They're not driven by an 8% total share of the student population. They're ESPECIALLY not driven by an INCREASE in a demographic that (at very, very generous estimate) tops out at 8%. Like, suppose population of school-aged, undocumented immigrants in the U.S. literally *doubled* from 2019 to 2024, from 5% to 10%[1], you'd still have to argue that an absolute change of 5% in the total student population was responsible for those massive changes in the graphs.
So unless you're going to try to sit here tell me that somehow 30-40% of the student-age population is undocumented immigrants with poor English skills (across every state, somehow), the argument above is bunk. At best it's misunderstanding driven by poor statistical acumen. At worst it's an attempt to shoehorn your hobby horse into an unrelated topic.
[1]Where we are generously adding an extra 25% for demographic skew.
I don't know what proficiency means but I hear a lot of people around me using a hybrid of English and Spanish. Less an insertion of words into a given sentence, though obviously there is some of that especially with newer or vocational-related terminology, than a flowing back and forth within the same conversation, perhaps reverting to the one or the other when speaking more quickly - whichever is a little faster and more comfortable. I think this is unconscious. Now, I know that being "bilingual" is considered a praiseworthy sign of genius and virtue - but my impression of this hybrid is that it is a cruder version of both languages. I enjoy learning new words in Spanish, for instance, and Spanish speakers naturally as any of us would, enjoy being appealed to in the role of master; but I often find that the person I'm asking does not know the word any more than I do.
This may be the way you make the US Mexican, and is not unexpected; but it probably isn't making for better reading literacy.
the high-performers have inexplicably great years from 2017 - 2019, but then return to average?
——————————————————-
Agreed that the data is all wonky, but this one at least strikes me as having an obvious answer: high-performing students are much more likely to have parents who are paying attention, and a lot of those were aggressively home-schooling during the lockdowns. Then those kids went back to the public schools. Oops.
Bullies may or may not disproportionately target high-performing students (though my money is on not; the relevant feature of "nerdiness" here is social isolation, not academic ability) but it would need to be *absurdly* disproportionate to produce such a large effect on high performers and be undetectable otherwise.
I home schooled my kids during Covid and my older son got through 4 grades of math in one year. My kids got more teaching during the pandemic, not less.
I question the premise that chronic absenteeism is due to not bothering to show up. I think it's due at least in part to more illness. Covid-19 as a thing you can get sick with has been added to all the other things you can get sick with. And chronic absenteeism is missing just two or more days per month according to that AEI report, which could easily happen with one febrile illness.
You've nailed it. The school where my spouse works always had the policy "if your child has had a fever in the last x hours, keep them home." But they were very lax about giving that message to parents, checking temperatures at school when kids seemed sick, calling parents before the end of the day to pick up their feverish children, etc.
Since the school reopened post-COVID onset, they've been very diligent about promoting and enforcing that policy. A kid who has a minor fever on Saturday evening but is fine on Sunday morning is expected not to be at school Monday.
COVID can and does cause long-term immune damage. Hardly implausible that people are actually sicker more. Anecdotally, I have definitely observed that.
"people are sicker more" - sure, that's plausible. But twice as sick *on average* is an enormous effect, especially when you consider that that average includes the vast majority of people never report any long covid symptoms at all.
Does that happen a lot in children? Does it happen often enough to make them sick twice as often as before, in numbers large enough to affect the statistics?
One illness might make two days extra of absenteeism. But I would be very surprised if there's a significant number of children that average one illness of this level of significance *per month*. There would have to have been a lot of children who were just one month short of this threshold for one additional annual illness to have pushed them over this threshold!
I don't think you have to be chronically absent every month to count in the statistic. Any kid that has missed two days is chronically absent for that month. You can be back in compliance the next month.
Edit: the AEI definition is annual, 2+ monthly absences increased from 15% to 28%. The NAEP definition is monthly, 11% to 22% but for 5+ missed days. Bit hard to make sense of, actually.
This is likely part of the effect. If both parents had to be in the workplace, absence of the child due to illness was probably serious. One of the parents has to take time off.
If the child can be at home because one of the parents is there, then the criteria for the parent pulling the child is probably lower.
If there's a hybrid arrangement and "the kid can listen in on the app" the criteria likely gets even looser. Although I suppose that counts as "present", so maybe that's not a factor.
No idea if there's more two-parent families now; that may be another factor.
Ah that’s interesting! So if the average child has Covid once a year, and is counted as “chronically absent” in the month they have Covid, then we should expect “rate of chronic absenteeism” to go up by about 8% post covid compared to before. (Maybe more if COVID infections are concentrated in January, February, and March, compared to summer or winter break; maybe less if a significant number of kids go more than a year between covid infections that are significant enough to take them out of school for three days.)
That seems like a really odd measure for "chronic absenteeism". It doesn't measure what people think it does.
OTOH I can see this being approved by a committee as a compromise metric. "Compromise" being defined as "stupider than any of the originally proposed metrics".
I looked at the scores at: https://www.nationsreportcard.gov/reading/nation/groups/?grade=4. Interestingly, there is a uniform decrease across most categories, by race, by school type, by region etc. The only exception seem to be the Asians, who have seen their scores increase. Not sure what to make of this information.
This is anecdotal but I felt like during the Obama years the dominant view among Asians I knew is that we’re all going to assimilate into a post-racial mass and it’s okay to accept regression to the mean because we’ll be fine, as expressed by this article pretty well: https://www.nytimes.com/2018/06/22/opinion/sunday/asian-american-tiger-parents.html
After Trump and especially after the anti-Asian COVID hate the feeling is more that regression to the mean is dangerous since we won’t really be securely accepted as equals and so we have to still push our kids to be the best they can be so they have in-demand skills to take care of themselves. The author of that above article himself was recently blocked from a judgeship despite having pretty strong qualifications, and I think the views he expressed are clearly minority views among Asians today.
Maybe there is evidence that relative Asian test scores started increasing faster in recent years than before.
"especially after the anti-Asian COVID hate the feeling is more that regression to the mean is dangerous since we won’t really be securely accepted as equals and so we have to still push our kids to be the best they can be so they have in-demand skills to take care of themselves."
That time period correlates with Asians becoming more likely to vote Republican.
Makes sense considering who was perpetrating the majority of anti-Asian violence and who was benefitting the most from anti-Asian discrimination in hiring and admissions.
From my experience with kindergarten and 2nd grader in a "good" district, the standards and expectations were lowered somewhat in last couple years, likely a DEI effect.
I suspect it's more likely a "dealing with the post-covid chaos" effect than a DEI effect, especially if it's just in the past few years. DEI was quite significant in the 2010s as well, if not more so!
One thing I didn't mention on the post but just thought of:
Lots of concerned parents didn't like lockdowns and put their kids in private/home school. Presumably these parents are selected for some kind of positive train (caring about education, having resources) so their kids probably do better than average. If many of these better-than-average kids stayed in private/home school after the lockdown, that lowers averages.
Assuming the first Google search results are to be trusted (which they might not be) homeschooling rates in the U.S. are only somewhere in the 3-6% range. It seems very unlikely that removing ~3% of students from the dataset would have this kind of noticeable impact, even you picked specifically the top 3% (which of course homeschooled kids won't be).
1. It would be great to see comparisons to other countries. Not seeing them is sadly a very common occurrence on blogs by US-bloggers, even otherwise very good / rigorous ones like Scott.
They would help disprove hypotheses like "this is because of wokeness" or "something something republicans making minorities uncomfortable at school something something" (both unlikely, but likely to be brought up in online arguments because culture wars are a thing), as well as stuff like "the tests changed"/ "now it's harder to cheat" / "students use computers instead of paper and that's bad" / "some overlooked legal change removed teacher incentives to teach to the test" (a lot more believable).
2. How come nobody is mentioning smartphones? There's plenty of scientific evidence at least pointing in this direction, and there seems to be a notable uprise in short video / Tiktok use that coincides with this timeframe.
See my comment elsewhere in the thread for a comparison to Australian NAPLAN data; the tldr is that the dip doesn't exist in Australia, so covid-related theories are mostly off the table (and US-specific culture war theories are still firmly on it, so go nuts).
Didn't Australia have a *very* different pandemic than we did, though? It seems at least plausible that the lowering of behavioral and academic expectations that anecdotally seems to have happened here wouldn't have happened there because our cultures interacted so differently with COVID.
Yes! I was internally screaming this while reading the article - if you want to compare the effects of different policies and disentangle that from culture etc, there's a whole world out here with a lot more variation than just the 50 US states.
Made even worse by the final chart, from TIMSS - an international test covering 50+ countries each round. You have to make a conscious decision to ignore the international comparisons and just pull out the US data.
Excellent point. Trying to decipher trends by looking at only one country's data is like trying to understand why chess moves were made when only looking at one color's pieces. When AI gets slightly better, I'm going to have my AI assistant remind me to check international comparisons every time I see a discussion of trends in the US.
In the meantime, I had Deep Research look into it. Preliminary results (i.e., me not confirming its output) suggest the negative trend is international. It called the drop in OECD countries’ PISA scores from 2018 to 2022 “unprecedented” and "scores revealed that many countries… fell back to levels from a decade ago."
“In short, math proficiency declines have been broad and substantial, affecting both advanced economies and developing countries. No region was entirely spared, though the severity varied.” Reading scores also declined, but not by as much as math. This is a consistent pattern (speculated to be because reading is easier to do independently and remotely).
As another commenter mentioned, it also notes Australia is an exception (along with a few other countries).
“In summary, the decline in U.S. test scores is not a uniquely American phenomenon.”
It also blamed the pandemic and the pandemic responses (unprompted). It says “countries with prolonged school closures tended to suffer larger learning losses…. A global study of TIMSS 2023 data confirms a strong association: nations that kept schools closed for only a few weeks generally stayed closer to their pre-pandemic achievement trends, whereas those that closed for many months fell far behind.”
It also notes some differing opinions, such as OECD director of education Andreas Schleicher, who said, "COVID probably played some role but I would not overrate it. There are underlying structural factors, and they are much more likely to be permanent features of our education systems that policymakers should really take seriously." I did confirm this person exists and the quote is real.
"The Programme for International Student Assessment (better known as PISA) 2022 saw an "unprecedented drop in performance" across the OECD regions. Compared to the last edition, in 2018, the mean performance in reading fell by 10 score points, while maths fell by nearly 15.
This drop in maths was particularly apparent in countries like Germany, Iceland, the Netherlands, Norway and Poland, says the OECD, which all saw drops of 25 score points or more."
The fact that scores have not only not recovered, but continued to fall is particularly striking. I would naively assume that the rate of change for learning is higher if your learning has been previously slowed (it’s easier to catch up than to learn at your age appropriate rate). But even if the learning rate wasn’t higher, I certainly wouldn’t expect it to be worse. Yet if the scores are still falling, then that suggests the negative impact of the COVID years has compounded in later non-COVID years such that today’s 4th graders now are worse off than those who had just exited COVID in 2022. To me, that does suggest something structural happening (or I have a very incorrect mental model of learning).
It makes sense to me that learning has critical periods that are hard to recover from, both neurologically and in terms of standardized curriculum.
IE, if you hit the 'bad' patch of schooling at an age where you are supposed to be learning the fundamentals of literacy or reading comprehension or mental arithmetic or etc., then it makes sense to me that this is very hard to recover from later.
Especially so if the teacher is forced to follow a curriculum that assumes you have already mastered those skills, instead of noticing your deficiency and changing the plan to focus on correcting it.
I think many people and even your initial post hint that schools will work hard to catch up for any learning loss they see, and that there is both a lot of slack (in the wastefulness of a school day) and enough institutional drive (incentivized by test score monitoring) to make sure we get things back to where they were initially. But remember that everyone is health sensitive now. So any signs of sickness kids are much more likely to be sent home, and so any claims of (real or feigned) sickness also result in being home.
Louisiana was sitting near the bottom of education ranking/results and instituted a number of reforms just last year. These reforms resulted in them going from 43rd to 32nd place.
My guess would be the teacher training reforms. As of 2021 Louisiana requires elementary school administrators and K-3 teachers to complete "a foundational literacy skills instruction course that is based on the science of reading ... and includes information on instructing students regarding phonemic awareness, phonics, fluency, vocabulary, and comprehension." Mississippi did something similar a few years prior, and saw significant improvements as well.
And of course it also helps that they were both performing very poorly to begin with.
I don't know if this has any explanatory power, but I'd like to see a version of this analysis where, rather than comparing a raw difference in score 2019-2024 by state, you look at a deviation from the trend--are there certain states that had been declining on average, pre-COVID, and then others that had been steadily improving?
I blame the proliferation of short form content, and primarily TikTok.
I don't know how many middle-class suburban American families you all know, but of my extended family who fits the bill, a concerningly high proportion of the male children are addicted to (what is essentially) softcore short-form porn, and generally attention-killing content. I know this has been an ongoing problem and accusation agains the internet more broadly, but I think this time it really is "different."
I really don't know what effect this would have on grades (which I assume correlate to my made up metric: "attention span"), but I can't imagine anything good. The drop in grades approximates the proliferation of TikTok, Instagram Reels and Youtube Shorts.
And as long as we're blaming new technologies, turning in assignments written by AI is, as I understand it, completely rampant and nearly impossible for the average teacher to combat.
'Lowering standards' is one explanation for why grades are staying high while standardized test scores are falling.
But 'all homework is done by AI and gets As, everyone flunks in-person tests because they were using AI instead of learning' also accounts for the same trend.
AI writing (at least the kind I run into online, or generate) is not at all good at disguising its voice, and it's not at all rare for a forum post to be greeted with "fuck off chatGPT"
Are elementary/middle school aged kids using the free models really good enough to disguise that voice but not good enough to just answer the questions correctly?
The transgressive thrill of cheating is not to be underestimated. People may get a sense of satisfaction from fooling their teachers that they might not get from merely studying the material.
LLMs have very distinctive voices, but weak human writers generally *don't* - one bullshitting high schooler sounds pretty much like the next. If you're not already attuned to GPT-voice, as I suspect most people are not, it probably sounds like a bullshitting high schooler too - especially when given high school essay prompts. The entire student body mastering English grammar all at once probably *should* set off some alarm bells, but if you're not already looking for it, would you notice? I doubt it.
I believe this is an example of confirmation bias. We tend to notice when someone uses AI because of its distinctive "helpful assistant" tone, but we often overlook its more nuanced use as a supplementary tool—rather than simply asking it to "write a whole book report."
accusing a student of cheating is very serious, implicitly the standard is "beyond a reasonable doubt". watermarking and AI detectors don't actually work. so it's usually pretty obvious but also not conclusive enough to throw the book at someone.
especially as even honest kids may have their writing style influenced by LLM outputs that they read.
Well, remember Scott's test of ability to tell AI from human in terms of generated images. Some people were good at the task, but 'good' at the 80% range, not 99.9%.
If you are a teacher deciding whether to give a student a zero on their assignment and refer them for disciplinary action because you think they used AI, are you ok with being wrong and doing that to an innocent kid who did their own work 20% of the time? The way parents complain these days, I think you'd get fired pretty fast if you insisted on that policy with that accuracy rate.
How much does the stuff kids do in school matter, as compared to the stuff kids do at home? Anecdotally, as a college student, I've found that my outside-the-classroom activities have way more of an effect on my reading/writing abilities than anything I do in the classroom. Getting outside, seeing friends, not spending too much time on my phone, etc-- these things all have an effect on my mental concentration and performance that is 10x the effect of any specific assignment I do in the classroom. If I e.g. spend 10 hours on social media on a Sunday, then can't concentrate the next day, that has nothing to do with my teacher.
After the pandemic, kids have been vastly more online than they used to be. If a kid is going home from school and playing video games for 8 hours, no amount of fancy classroom management or test prep can counteract that. My take is that kids' home lives have changed (most importantly: more time on smartphones), and that's what accounts for declining test scores.
In 2020-21, I worked at ETS as a user experience consultant on the NAEP. The major problem we had to solve was to maintain long-term trend while transitioning from a device-specific digital assessment (i.e., the NAEP was being administered on locked-down Surface Pros that the DOE shipped to school districts) to a device agnostic digital assessment (i.e., one that could load in any browser).
Psychometricians are supposed to be norming the test based on usability changes that might affect scores. Digitization poses a lot of variables that can confound long-term trend. I wrote an internal report about the effects the "digital divide" has no test takers that need to use assistive technologies. Basically, low-SES kids can perform even lower on digital assessments than they would on paper-pencil because they, for instance, may not recognize the icon that opens the calculator tool as they are less familiar with digital cues.
Anyway, I don't have much data to share but I thought I'd point out in 2024, the device agnostic assessment launched on Chromebooks. For all we know, the test wasn't normed very well against long-term trend and could have usability issues affecting it. As the timeline shows, the test was paper and pencil until 2017 (for math, reading and science), which is when a lot of these oddities you noted started happening.
I could imagine that part of the effect is due to differences in the test from one year to the next. Particularly because a score decrease from 216 to 214 seems small, in both relative and absolute terms. It appears larger in these graphs where the y axis starts at 200.
On the other hand, other tests in other countries show similar trends.
one interesting result. Maine and New Hampshire are right next to each other. Maine is the worst. New Hampshire is on of best. This could be a cause of investigation. Why do two schools just over the State Line have wildly different experiences?
"Absenteeism" is also happening when school is just not running classes when they're supposed to. Ever since COVID, my local school district has a number of "asynchronous learning days" on its calendar. These count toward the state-mandated number of school days for the year... but they're a lie. They are 100% days off from school. Maybe some kids are doing some homework or something, but there is no class happening in any way any different than a holiday.
Even rigorous schools in my area have become very comfortable with calling off school in a way that was unheard of 10 years ago. There's a private school nearby that had a reputation for being open unless the door physically wouldn't open due to snow drifts, but since 2020 it's fallen in line with other schools. This year it closed due to a forecast of mild snowfall that didn't even materialize.
A really obvious way to check the COVID-related hypothesis, especially, would be to compare to data outside of the U.S. Obviously you won't be able to compare test scores 1-1, but it should be easy to see if a given country's benchmarks-of-choice show the same sort of sudden drop. If it's much less common outside the U.S., that points to something specific about the U.S. education environment. If it common among many countries, that points strongly to COVID. In the latter case, asking which countries were least effective and what they do/did differently may prove fruitful.
One would need to be careful to control for differences in vaccination rates, infection incidence (for example, differences in mask-wearing and ventilation), etc. I don't know how possible it is to control for this stuff, other than interventional (or quasi-interventional) studies done within individuals, which doesn't really give as much insight into population level trends.
That would be necessary in order to draw specific, robust conclusions, sure. But the point here is mostly to do a quick sanity-check on the main hypothesis (and perhaps to locate plausible alternatives if it fails). The thing Scott's looking to explain is a very large, sudden drop in test scores. If there is no corresponding large, sudden drop in test scores in a significant fraction of broadly-similar countries, the answer *could* be [complicated statistical reasons that made it show up across all U.S. states but hardly anywhere else]. But it's *more likely* to be "this hypothesis is wrong and there's some alternate cause instead." The signal in the U.S. is very strong. It's possible but very unlikely that the same cause could apply to many other countries, but get masked by noise in almost all of them despite being strong in the U.S.
Speaking as a math tutor, I've seen dramatic effects from school closures. Anecdotal, I know, but apparently the research backs me up.
In fact, for some of my students just skipping school entirely might've been better than online classes were. At least then they wouldn't have been taught to cheat.
Looking at similar tests in similar countries could help eliminate some possibilities. The NAPLAN tests in Australia are equivalent; if it's Covid-related then there should be a similar dip:
There's no covid dip. (There is a dip in 2023, but this is attributed to the test being moved two months earlier in the school year.) So I'm inclined to eliminate covid-related theories, whether due to the lockdown or the disease.
having been to both australian and american schools in elementary, middle, and high school (albeit decades ago of course), I'd say that there's really different norms between them in general: way more private / religious schools in Aus; all have uniforms; way more sex-seperated schools (I went to an all girls high school); school sizes / class sizes much smaller; schools start an hour later, much more emphasis on rote learning; middle schools dont exist (high schools are 7-12, primary school k-6); compulsory schooling to year 10 only; trade school after year 10 are normalized; college major choice is restricted/determined by end of high school exams scores. In terms of phones, Australian schools were much freer about discipline and restriction--there was no hesitation to take phones or send people out of class. The whole system is radically different in my recall, but I'd be curious to hear from someone who went more recently.
Very state-dependent. Western Australia had one of the shortest and least disruptive lockdowns in the world (they eradicated the virus in a month and then went back to life as normal) while Victoria had one of the worst (because they were intent on eradicating the virus but kept failing). Melbourne had a total of 263 days in lockdown, supposedly more than any other city in the world, although I'm not sure how many days the schools were closed.
Anyway, the two biggest states have about half the population of the country and they both had extensive lockdowns so if lockdowns were a big issue you'd expect to see it in the NAPLAN results.
Also you'd expect to see a strong state-by-state effect where Victorian students got a lot worse and Western Australian students didn't. But we don't see this:
Australia barely locked down aside from Victoria, and as you say the scores *did* drop in 2023 (for confounded reasons). The by-state data you've linked only starts in 2023, so we can't see a time series for how Victoria *changed* relative to WA or SA (the least-locked-down states).
Also, additional confounder not mentioned - NAPLAN testing was absolutely FUBAR'd by technical problems the last few years. Stuff like students seeing the test, not being able to submit results then re-taking the test a few days later when they've had days to think about the questions. The transition from paper tests to online tests happened over several years from 2018-2022.
I don't think any useful conclusion can be drawn from this data.
I remember reading your earlier article and thinking "what applies to one child missing school is not likely to apply when all the children miss school together" -- there is probably a swimming-with-the-current effect when you jump back into a peer group that's made progress while you were away. There's pressure, and help, to get back to your (known and obvious) place within the group.
I think the vibes argument also applies: though it's merely anecdotes, there were a lot of news articles in post-COVID educational journals about how students seemed much more checked-out, stressed, disruptive, and distractible after the pandemic. You could imagine the pandemic -- both its immediate effects and the secondary knock-ons, ranging from increased mistrust of schools, angry woke teachers, angry anti-woke parents, etc -- diminished the overall level of motivation in schools, both for teachers and students, in a way that is really hard to quantify because just applies a small negative effect to everything across the board. The absenteeism issues might just be one small manifestation of a much larger "school bad" vibe.
Also: notable that the "best" year in most plots is ~2012-2013; that was the last year where <50% of Americans owned smartphones.
you’re overthinking it. If you were a parent during this period you saw the devastating impact that the Covid shutdowns had on schools and kids. It wasn’t just missing some classes it was a cultural shift. As some people said below it was in some ways a continuation of existing trends - but it took all the worst existing trends and supercharged them
I suspect a combination of lower standards and worse social behavior standards. I have a son-in-law who quit high school English teaching a few years before Covid hit - and from what I have heard, his peers who stayed teaching have found it far more frustrating and challenging than before.
But the point I want to raise it that the schools tend to think that they are teaching their students and that changes in their curriculium and methods are responsible for changes in the learning levels. And this may be true to some degree, but highly educated / professional parents are at least as well educated as the teachers and are likely to have higher standards and expectations for their children. And they are highly likely to supplement their children's education with their own guidance or paid supplementation. In either case, the education of these children diverges from that of their peers who are not having their education supplemented.
I spent hours flashcarding math facts into my kids - and reading to and with them until they mastered reading and starting reading for knowledge and enjoyment - which meant very limited screen access. I also supplemented their math education - mean dad, bad dad. But they mastered the material.
I can easily believe that the Covid wave forced more parents to be active in their kids education and some parents in good disctricts may have slacked off - doing all that education work takes a lot fo time and effort.
But kids who rely solely upon school for their education are going to be badly impacted by lower standards, worse social behavior, and addicting screens - short videos and the social network social games are far more addictive and diverting.
My kids hated my intrusiveness when I was overseeing their education. But they are following the same pattern with their children, and discuss approaches with me.
I have a couple good friends who are or were teachers and it does seem like the teacher burnout from the profession becoming increasingly regimented to state curricula with little autonomy and the general rise in antisocial behavior among kids and harsh discipline enacted to counteract it, such as the regular lockdowns that happen at a middle school one friend is at, accounts for most of this.
One thing to look at could be private vs. public schools. Many private schools went through the same “wokeness” that public schools did, some even banning advanced classes and tests, but generally also give teachers a lot more autonomy so they aren’t burned out and have smaller classes and softer disciplinary policies. Did student performance in private schools also drop or only public ones?
What do NAEP scores show about the persistence of changes in performance? Is there any clear relationship between 4th grade scores and the scores of the same cohort in 8th & 12th grades?
Something we must not forget is the long-term general trend of grade inflation. We've all come to take it for granted. But there are no grades higher than A, so this inflation cannot go on forever. I know an admittedly small number of children, so my sample size is small. But the kids I know are "smart" kids in the sense that they can figure stuff out, can crack clever jokes, and have a lot of common sense. They are curious, adventurous, and great at solving puzzles in video games. But even by age 9 or 10 they can't tie their shoes, spell out their entire name, give you their address or phone number, or multiply two single-digit numbers together. And the kids I do know, attend American public schools and an Australian private school, so it's an international problem. What's going on? I think it's just that parents don't do any educating at home. They shrug and send their kids off to school. Schools do nothing, kids are bored, administrators and teachers all get a paycheck, everyone goes home at the end of the day, and it starts all over the next day. At the end of the semester they get report cards with all A's. Surely this can't continue through college? Oh, but it does. As long as they don't major in STEM, they'll be fine. They graduate, fail to launch, and blame "capitalism" for their failures.
I am a long time reader of Scott Alexander. This is my first comment ever. I hope he sees it. :)
The No Child Left Behind Act included both requirements that schools measure student performance in various subgroups (such as low income students) and strong incentives for school districts to raise student achievement. Failure is often rewarded in education policy—such as more funding for low performing schools. But NCLB actually had economic incentives for schools to perform well; schools that were failure were required to pay for tutoring by outside entities. So student performance improved both because of the tutoring AND the incentive schools had to avoid paying for it by increasing their test scores.
When congress ended the No Child Left Behind Act, it was only a matter of time before test scores plummeted.
Do we have any research about NCLB? The Center for Educational Progress calls it the "worst education policy in decades" (https://www.educationprogress.org/p/schools-should-pursue-excellence). At the same time, that center has a specific agenda. I'd be curious to hear well-argued pro-NCLB points.
On the other hand, many commenters have pointed out that similar trends in test scores can be found globally. This would point at a more general underlying cause, not a US policy.
I just pulled up a graph of Ontario test scores in math and the trend from 2000 to 2018 looks very different from the USA.
I made the best argument for NCLB I could in my previous comment. Millions of students in failing schools got tutoring from private tutoring providers. This raised scores for two reasons. First, tutoring in reading and math is very effective. Second, school districts wanted to avoid paying for the tutoring so they focused on raising test scores for sub-groups of students who were normally ignored.
I am very grateful for your reply Jonas. Hopefully Scott Alexander will weigh in as well.
I am not an American, so all my information is second-hand, but I thought there was a consensus that NCLB made it so important for the least performing students to improve, that everything else (i.e. both the average and the best students) was sacrificed for this goal.
And then it turned out that the easiest way to slightly increase the results of the least performing students (because there is no known way to increase them dramatically) was "teaching to the test", which made things even worse for everyone else.
If this is true, then giving up NCLB should result in the least performing students getting slightly worse results, but everyone else getting much better results. So either this is not true, or it takes more time for the American schools to return their focus to the average and the best students.
I don't know about grade school, but at the university level (where I teach) there has definitely been some inertia in COVID-era policies that stuck around even after lockdown, analogous to the "stopped enforcing tough-on-absence policies" thing you mentioned.
The most prominent example I have seen is the removal of standardized testing: SAT/ACT for undergraduate admissions and GRE for graduate admissions. Lockdown is long past; in 2025 it is equally straightforward for a student to take these proctored exams in person as it was in 2019. Yet the tests are still not used.
Many such practices that started as "we need to do this, even if we don't want to, because with remote classes and remote work, we simply have no other choice", nevertheless continue now due simply to inertia.
But in some cases they continue because someone with an agenda used COVID as an excuse to enact the policy and now wants to maintain it post-COVID. They realize that it's easier to defend the status quo than to change things, even if the current "status quo" only just came into being during lockdown as an ostensible temporary measure. In other words they seized upon the COVID crisis to shift the Overton Window.
An example I could imagine in pre-university schools is this idea I had seen prior to COVID, but it was rare: the concept of setting a minimum of 50% for any grade. (see https://www.ascd.org/el/articles/the-unwinnable-battle-over-minimum-grades for example) I knew a high school teacher who worked at a school where this was the policy, and it was controversial among the teachers for obvious reasons. I have to imagine this policy is more widespread now due to COVID, and once in place, more difficult to roll back than it was to enact in the first place. (Just a guess though, I have no data on that.)
1. Teacher pay especially relative to cost of living, has declined since 2010, and the epidemic was likely just a wake up call to those on the fence that “essential worker” was essential the way slavery was essential to the antebellum south
2. Goodhart’s law predicts metric collapse, as the system tunes to a metric and eventually fails. It is possible that schools cut everything that didn’t seem to support the metric, then cut a few tendons too many
3. You have to factor in devices. Every teach I know says attention spans are nonexistent. You cannot read or do math if you can’t focus on symbolic decoding or think
4. The runaway 1% has increased anxiety among professionals and that has to include parents. Stress does not equal success
COVID can damage the brain. You don't think you're possibly seeing the result of letting a disease with systemic consequences rip and damaging children's brains?
"I predict that what we’re seeing here is not each individual child’s learning loss multiplied across all children, but a systemic effect where something about the pandemic made schools worse"
Or another possibility: schools' problems predate COVID and have continued. As you noted, the national trend started in 2017-2019. In 2014, the Obama Education Department issued a "Dear Colleague" letter that advised school superintendents that racial disparities in suspension rates would be grounds for finding school districts in violation of Title VI of the Civil Rights Act. News reports suggested that many schools responded by dramatically reducing efforts to enforce any discipline standards. To the extent this is true, it would make it harder for teachers to teach, at least in some districts with significant disciplinary problems.
Another item: it has recently been published that Connecticut schools have an explicit practice of never making any student repeat a grade for any reason - students don't have to learn anything to move on to the next grade, and eventually high school graduation. There was one notorious case last fall where a Hartford High School honors graduate sued the school because she can't read or write. I don't know to what extent this issue is reflected in other states, but it would support the idea that standards have declined, but not because of COVID.
I interviewed several teachers in 2021 and 2022, and nearly all of them told me that COVID had delayed not just their students' academic development but their social development and general maturity. One said "my 9th graders are really 7th graders". I've read others saying something similar. This would be an argument that the scores will recover, and that they should recover first for the youngest test-takers, then continue as they get older. However, the downgraded norms are being sticky; teachers who allow more misbehavior than they used to (they all told me they do this) may not realize they can raise expectations, and they'll keep getting students who have had other teachers who also didn't realize they can raise expectations, so they can't even raise expectations as much as they'd like (unless they have much more support for high expectations and consequences from parents, administrators, and the public)
This! Absolutely seconded from what I have heard from teachers in Germany as well. Only anecdotal, but they are very convinced that the students are less mature and less socially developed. I think all academic decline is downstream of that.
I am not so sure that this is *only* the lockdown. I think there was already a trend in that direction before Covid, at least in Germany. But the Covid period where kids are not socialized in a group (a long period, in Germany) has accelerated the trend a lot.
Yep, COVID leniency has been sticky. One of my local high school districts made 50 the minimum grade on all assignments and exams. You can literally do a third of the work and still pass.
Also during Covid BLM hit. School discipline declined because of equity, school to prison pipeline etc. Hard to learn when you cannot get rid of the badly behaved children.
Social media and Chat GPT are the primary problem from I what see with children in Europe. COVID seems to have accelerated that. Children don’t learn how to read or write. Parents are also too distracted to get involved in their children’s education.
Interesting. I was wondering about this on international scale but hadn't looked it up.
Standardized tests in Germany suggest similar alarming trends of worse abilities. Experience surely does.
Not sure it's about COVID. Falling quality standards in schools is something I would also bet on from my personal experience as a parent of two kids.
That includes missed classes because of missing teachers (they are ill or their kids are)
And also focus on other things and minimizing (demonizing?) basic repetitive learning (I think it makes sense to some extent, but the result at this stage of education, ie 1st to 4th grade, is clearly bad)
Their school is considered quite good and my kids mostly like going there. But the basics are clearly lacking.
And I can't think of a parent who thinks to the contrary, also in other schools, so it's seems more like a systemic problem.
Not sure how much this is a mix of overparenting or self projection of our missed expectations onto our children... But we started taking those basics into our own hands for our first kid when we understood this and started directly with our second kid. And yes if course it's better.
Any possibility that COVID helped accelerate the saturation of internet and smartphone use, including normalizing putting a 3rd-grader in front of an internet-connected screen all day?
I don't know how localized this is, but where I live, there was a huge post-COVID wave of retirements and career-switches among teachers and other school staff, and since then there have been ongoing staffing shortages that show little sign of letting up. The staffing shortages lead to overwork and burnout among the remaining staff, and the rigidity of public sector wage scales means schools can't pay more to attract more workers.
Correlate it to the secondary, or recovery market price year over year for cellphones. I suspect another effect is that as old cellphones became cheaper the chance they were given to a younger child increased.
There was a very strong spike in recovery markets around 2019 just before Covid, and with Covid everyone had to get some sort of device to function. They may now be percolating to younger and younger children.
I think it’s either very young children are getting devices or specifically phones circulated around 2020/2021. Research has shown time and time again that before puberty children learn and retain the best from 1:1 face to face interaction with adults.
I gave my iPhones to my great great nieces and nephews as toys about 4 years after I upgraded.
Not hugely worried, but I did feel bad that I wrote a post saying that people would recover quickly from COVID learning loss and the test scores seem to contradict that, and I thought it would have been cowardly to ignore this and not explain where I stood.
Also, I'm not sure I would describe formal education as "all but meaningless". I don't think it's very efficient or compassionate or well-done, and I don't think changes at the margin matter much, but the difference between going to school at all and not is probably important.
Ah, that's fair. Was just jarring given some posts I... vaguely recall (don't have the links and all) on unschooling and such, though thinking may have changed in the interim.
I feel like there are a lot of people in the Rat community who don't think school is that important, and I think it's partially because they generally didn't enjoy school very much and have intellectual elitism because they're more intelligent than their peers. (I'm also intelligent and got bullied at school.)
I can't be bothered to express myself properly, so I'll just say something crassly: being socialised is important, and I wouldn't a priori expect it to be pleasant.
Yeah? You go from an environment where you are coddled and have very little competition (especially if you are the eldest or an only child), and enter an environment where you are in constant competition and need to find your niche and fit in while maximising your status and future returns. That sounds hard to me.
Socialization can be done in various ways. I think the pleasant way is introducing friends gradually, starting with people who are (sub)culturally similar, and gradually expanding the variety. The unpleasant way is to be suddenly dropped in the middle of a mob that doesn't share your values. School does the latter.
Well, we usually socialise children starting at age 3-5 where things are a lot easier. Young kids don't think much about status games and are usually happy to play with any of their classmates. Kids get much nastier as they get older, peak in horribleness around 13 or 14, and then get better as they approach adulthood.
The worst possible case would be to be home-schooled until the age of 13 and then dropped into a school environment.
There's overwhelming evidence from PISA, TIMSS and smaller national testing programs that test scores have been dropping gradually since 2015 or so (maybe a little earlier depending on the dataset and country). Admittedly, the trend is not visible in US PISA scores - I don't know why this is, but it's common to most European and English speaking countries. This site has nice visualisations but only includes data for 2018 - the next iteration is being published at country level around now, and should eventually appear there. https://www2.compareyourcountry.org/pisa/country/oecd?lg=en
The trends are visible across multiple subjects, though perhaps strongest and most consistent in maths. It's common across more or less all countries (some South East Asian countries excepted). Covid might have accelerated the trend a little bit but there's a greater trend there which is crying out for explanation.
Some possibilities - I don't have evidence for these but they seem plausible: widespread adoption of mobile phones by children and adolescents; reduced parental involvement in educating children; declines in the quality of schools/teachers/curricula; environmental & health factors like microplastics, time outdoors, Covid. I *don't* think that there's a single neat explanation - maybe all of these feature, some more heavily than others in certain cases.
Been waiting for this...not mea culpa, exactly, but prediction-scoring after enough data came in. Sometimes it's not clear how to resolve these sorts of claims, but the ongoing barrage of omnipartisan "acktually learning loss was pretty bad" news has had me wincing about that 2021 article for awhile now.
Be curious what the test scores are like at the collegiate level. My company doesn't employ enough highschoolers to get much useful data, but there was definitely a bump during covid of flustered students who took a leave of absence from dysfunctional schools to get an Essential Worker(tm) jerb. (Not because they loved working in grocery, but because so many of the other common undergrad side gigs like waitressing got shut down.) So that'd be a, uh...I think that's a compositional effect? Where students who left and then came back didn't change grade levels, and are obviously academically rusty, so test worse by default.
My guess, once I learned that learning loss didn’t seem to be correlated with school closures, was that the stress and trauma of living through a pandemic had a negative effect on children that showed up on test scores. Can we rule that out? I do agree that the hypothesis that the schools got worse seems more likely.
Economic issues during Covid were a huge thing, also, both in terms of household finances and also educational cuts.
But if you're trying to control for American policy changes, wouldn't it make sense to use international data to establish some kind of baseline for comparison?
Perhaps many students were statistically less performative because they found themselves struggling with being either oppressed or privileged or born into the wrong body? Plus, there has recently been the distraction of democracy itself being at stake, along with the looming threat of WWlll, as well as there having been a surge in non-native, undocumented students entering the pubic education monopoly, with some of these students being girls having to compete in sports with boys while simultaneously having to overcome a culture of rape. Then again, perhaps the findings are due to the radicalization of parents becoming domestic terrorist threats. At least students’ free speech rights are still safely protected from hate speech and micro-aggression, despite low test scores.
Originally I was flirting with "overall febrile disease burden, especially in the winter during these tests." We had those tripledemics after all... maybe it's not covid per se but the after effect of isolation and the vengeance of flu on immunity debt.
Or maybe just teacher enthusiasm, which surveys show has also dropped since 2019?
But the biggest problem with those theories is that when I look at the tests I'm seeing improvements on math and declines on reading.
These are divergent outcomes! Anything about changes in culture or kids or teachers or kids explains too much.
So I went to the BLS time use survey looking for some indication that our culture has become STEM-pilled. There are declines in reading for men with big jumps for computer use.
Still a bit of a stretch, the reading declines aren't there for kids in the time use survey, and "too many video games" doesn't match the divergent outcome so much.
I don't know what to think, especially with all the different assessments out there producing different trend lines. I guess if there are a bunch of assessments with different trend lines, maybe it's mostly noise and these will converge in another year or two.
Possibly related: "reading" as an activity becoming increasingly identified with primarily with 40s-ish women (source: the gift basket I won from my library for a reading contest over the winter, along with seeing what new books pop up on Overdrive/Libby)=reading becoming even less cool than it has been=kids wanting to read even less.
Of course, there's widespread reports discussing how now Zoomers are less and less literate and rely on video/audio for learning/and entertainment and are reverting to what looks more and more like an oral culture; if we assume that trend to continue (it's not like we're making less video/audio content these days, after all), one would expect reading to go down, which would also probably make people get worse at reading in general. Of course, the causality arrow might go the other way (people get worse at reading so they read less)
Indeed. There were two huge cultural events in the first half of 2020. The Establishment has been trying to memoryhole the Racial Reckoning since, approximately, later 2022.
Good private schools are way woker than public ones. Just go on the websites of some famous private schools, they all go out of their way to show many black students they have and emphasize diversity. Sidwell Friends, where our political elite send their kids, the top article on their website now is about LGBTQIA+ and two others are about the Black Alumni Alliance and students staging a classroom takeover.
You are confusing good private schools with elite ones. There's a huge difference. Good ones might have bowed towards the DEI altar, because they felt like they had to but continued to have rigorous education "counseling out" kids who couldn't hack it.
Elite ones prostrated rather than bow.... but then, top-notch education was never their only (or maybe even main) goal. Their goal is to produce members of the elite.
This result seems to be global, not just American, so there must be some definite affect from closing schools and the way return to school was handled.
"A new study from Mary Immaculate College (MIC) has found that current classes of junior infants may not be faring as well as children born before the pandemic, according to their teachers.
The research, ‘Infants of the Pandemic: Teacher Perspectives on the Early Development and School Readiness of Children Starting School in September 2024’, revealed that 81 per cent of teachers have more pupils now with emotional and behavioural issues compared to pre-pandemic times.
The online survey was completed by 107 junior infant class teachers from around Ireland (teaching over 2,000 pupils overall) during December 2024.
The survey aimed to find out if experiencing the pandemic during infancy might have influenced later development when starting school."
Junior infants is the age range 4-5 years of age; children are enrolled in primary school at age 4 in Ireland:
I think part of it is the same as the whole debate on here about "school is prison": for the bright, motivated kids and involved parents, lack of formal classes was less disruptive because those kids would continue to learn on their own/have parents making up the difference. Most kids would avoid doing homework, class work and the like if they didn't have to, and doing online classes was easy to skive off. Teachers had their own problems. By the time things returned to normal, the kids were accustomed to avoiding doing any more work than could be dragged out of them, the lost time wasn't caught up, and the accommodations (happened in Ireland as well) for less rigour around testing and grading just kept on sliding - after all, if you gave little Johnny a B for his work last year, why are you now reducing him to C or D when he is handing in the same level of work now?
You should probably talk to some school teachers, because they all seem to have a pretty good handle on the thing that's confusing you from your desk jockey chair. From the perspective of multiple high school teachers in Georgia, they identified three things with 2020:
1) the school lockdown nuked a year worth of learning progression, which is what you were talking about, but also
2) the BLM movement installed a lot of school board members and therefore administrators that were very much NOT about discipline in schools because (systemic racism) which led to the inmates running the asylum - higher rates of fights, disobedient kids, kids taking advantage of a system that would never punish them for bad behavior, and so forth,
3) cell phone addiction skyrocketed because parents used the cell phones as a babysitter during 2020 because they had no other choice.
So it was three awful things, not one, and two of the three haven't been unspooled yet. The process of unspooling them has begun, but cell phone bans only started to hit a few public schools in 2025, and they're difficult to implement, and BLM oriented school boards have to be voted out which takes a while, and in some areas will never happen.
Want to emphasize my agreement with (2)... The school I attend is in a fairly lefty area, but its administrators and teachers are even further to the left. (My understanding is that this is pretty common in K-12 education across the country.) After Covid, behavioral issues became far more common, and discipline became far less strict and far rarer.
Nowadays, if a middle schooler curses out a teacher or gets into a fight, they go through a "restorative" (fifteen minutes of saying they're sorry to the teacher or to the other kid while an administrator watches) and then... usually nothing else. Maybe a particularly bad repeat offender will be suspended for a day or two.
Scott wrote about academic and attendance standards slipping—in my experience, the drop-off in behavioral standards has been even more extreme and is probably closely related to lower scores.
I saw this a couple decades ago, when a counselor at our local elementary interposed mediation between a misbehaving kid, and the teacher who attempted to send him to the principal so that class could carry on.
I felt at the time this was an unfortunate equalization of the parties involved, and of course granted the kid power. What you describe sounds like postmodernism run amok; the kid will only feel that he’s been elevated, made important.
I wonder if this *ever* works as well as paddling once did.
Part of it, again, is that the worst parents are the ones who will scream the loudest about "you are picking on my little Johnny!" if the school attempts discipline.
They don't care if Johnny is learning anything, if he's a troublemaker; all they care about is that they aren't bothered by anyone, and certainly that they don't have to have Johnny at home during the day when they'll have to deal with him themselves.
"3) cell phone addiction skyrocketed because parents used the cell phones as a babysitter during 2020 because they had no other choice."
That's a minor thing arising in my country now, because the Department of Education is going to spend/spending money on special smartphone pouches so pupils won't be distracted in class (why the school can't just take the phones off the kids and put them in a cupboard until going-home time, I don't know):
There are lots of disparate things going on in the USA to address this as well. Some are seizing phones, some are throwing them in faraday cage bags, etc.
> why the school can't just take the phones off the kids and put them in a cupboard until going-home time, I don't know
When phones inevitably disappear, or are alleged to have disappeared, who will be responsible?
Though personally I think it would be simpler and cheaper to have a policy of, if we see or hear your phone, then it is permanently confiscated. Maybe you get a warning the first time. The school can sell the confiscated phones to raise money for school supplies or whatever.
Yeah, but if you're expecting the kids to put their phones in a locked bag and then carry that bag around with them, some of them are going to try and break the locks, or they'll lose the bags, or it'll be "my phone was stolen at school".
We need better methods of measuring reading proficiency. The low test scores could simply mean that students are worse at taking tests (less motivated, less focused etc.), not necessarily worse at reading or comprehension.
I was a middle school English teacher when Covid hit. In March of 2020, we suddenly went remote and stayed that way for the next 1.3 school years. The way I think of it is that Covid remote teaching broke kids brains because they peeked behind the curtain and can't unsee what they saw. Imagine being in a relatively orderly, functioning classroom one day, with a whiteboard and desks and other students around you, and the next day (and seemingly forever) you're then in your bedroom and you can turn your camera off and you can play games on the computer and you can pretend your audio is not working and you can't hear.
The dynamics completely changed. Kids realized, school as I knew it is not a fact of life. Learning was mediated by one small flat Chromebook screen that made school 100X more boring than it already was. Sleep habits, already book, took a nosedive. You're doing everything in one space. Your bedroom is your classroom so the vibes of sleep and learning blend together. You get used to doing things your own way, being able to mute everyone, not having to participate... That is not good for mental health. Plus, young kids on computers are getting poor digital habits early. Also not good.
It was a shitstorm and it's still unfolding. Just go to r/teachers or r/teachersintransition for a little look-see. I left teaching at the end of 2021, mid-year, to join my wife in her business. This was one semester into the first school year back in the classroom. It was complete insanity. Kids constantly talking, causing chaos in the bathrooms that no one could ever catch, doing next to no work, mental health of everyone bad and getting worse. I understand these trends have only continued.
School was already antiquated and so bad in so many ways. This just pulled the mask off so everybody could see — especially the kids. I love kids. I feel terrible for them. We have bureaucratic lock-in of a broken system that breaks more and more young brains, all the more so since Covid. Ugh. Yeah. Dystopian.
A lot of that is pushing back responsibility onto parents - if they were not/could not monitor their kids at home to make sure they were attending the virtual classes and keeping up with the work, of course the kids learned bad habits.
The school system as presently constituted may be broken, but the lockdown showed that without *something* in place, education will fall off a cliff as regards the majority of school-aged children.
One aspect not really brought up here or in your original post is that schools have a very different role for lower class children, where they serve as a key distribution point for most social services. Just as one data point, about 12 million children relied on schools for breakfast a decade ago. For this population, schools also fulfill more abstract needs, such as stability or functional adults. So school shutdowns should also be considered equivalent to shutting down most government support targeting children or families. It's possible that this had more of an effect than eliminating the actual pedagogy.
The only allusion to this issue in the original post I found was "if you come from a really bad household school is the only place you can learn expected values and behaviors". This issue and removing basic services such as food could account for a lot of the drop in scores. You do point out that there's conflicting data on low-performers specifically, but that's to be expected. Schools serving as social services hubs with pedagogy as a superficial side-benefit is inconvenient for most political narratives so is rarely even considered and the definitions and assumptions used in research aren't necessarily well-suited to show it.
How much is a 2-point difference in scores (from 216 down to 214)?
The difference looks significant in the state-level bar charts that Scott included. At the same time, it's only about 1% in relative terms.
Wikipedia says that "Main NAEP assessments are typically administered over approximately six weeks between the end of January and the beginning of March of every year." Part of me wonders if February 2024 happened to have particularly strange weather (It was the warmest on record). Or whether there is another unknown factor at play here.
I have another hypothesis to propose: current economic conditions make it impossible for parents to monitor their child's educational progress as closely as before. Covid, where shut downs put more families under greater economic strain, made this trend worse, but it started before then.
If correct, this implies that the school itself is less important than family circumstances. Certainly, families in impoverished areas are both exposed to schools that didn't have the resources to cope with the effects of Covid, but that isn't a perfect relationship. And it wouldn't just be poor families experiencing this pressure, but ones employed by businesses more greatly affected by the pandemic. I would like to see a chart for students who are the children of healthcare workers.
I remember my parents knew when I skipped a single homework assignment. They wouldn't let me out of the study area until I had completed each one, every day. I do the same with my children, but I wonder how common this is anymore.
Why? If by "Zoom era" you mean the shutdowns due to the pandemic, well, I was a parent of school age children then, and that simply isn't true. I was on my own zoom call, doing my job.
I can only attribute that to parents who previously had known nothing. And I can only attribute that to not caring. If you are the kind of parent who takes the time to become invested in your kids education, then in person classes aren't the barrier. The real barrier is the time and energy the parent has.
> Re-reading the post, I still think my arguments make sense.
This chart makes it seem like scores have been continuing on a pretty consistent trend since before 2019, with no change at all visible during covid. It's actually kind of surprising how little evidence there is of covid having any impact at all. Why would you assume there's anything going on here other than the continuation the pre-existing trend? This "trend" is based on 2 data points, true, but it follows a long plateau, so it could make sense if there's some longer-term trend slowly worsening education, and it's still the case that's there's really nothing to *explain*.
Our ability to build complexity has gotten too far ahead of our ability to live with complexity on net. We are slowly making our lives worse. Overall human performance is starting to degrade. Covid just forced a breaking point that was coming soon anyway.
My wife is a primary school teacher in Switzerland’s largest city, where there are many immigrant students. We have similar discussions here about the decline in reading and math skills.
Reading:
A key issue is that the language of instruction in schools is not the native language of many students, which naturally affects their reading proficiency. But beyond that, deep reading itself seems to be becoming an outdated skill. For most people, there’s little pressure to reach a high level of reading ability anymore—neither for daily life, nor for entertainment, nor even for many entry-level jobs.
As someone who has avidly read both fiction and non-fiction for decades, I’ve noticed a shift in my own habits. I now consume many books via text-to-speech on my e-reader app and get much of my information from podcasts and YouTube—sometimes actively watching, sometimes just listening in the background. Recently, I uploaded a non-fiction book to NotebookLM and listened to an AI-generated podcast discussion of it instead of reading it, because I found the author's repetitive writing frustrating.
I barely use Google to find and read through websites anymore—I just ask ChatGPT. Kids today do the same, relying on AI tools, TikTok, and podcasts. It seems that reaching a truly advanced reading level is becoming a niche skill, rather than a widespread necessity.
Math:
I’m less certain about the causes of the decline in math skills. One possibility: In the past, highly competitive Asian "tiger moms" might have pushed their kids toward U.S. degrees, but with the West losing its economic and technological edge, perhaps more of them are choosing other countries or staying in China instead. But I don’t know—just speculation.
PS: as a non native english speaker I drafted a raw version and let chatgpt do the rest...
I have definitely noted both a decline in written English (a lot of writing that spells words 'as they sound' and it's clear the person hasn't seen it written down, or at least their schooling never corrected it; one thing that seems to be becoming commonplace is using "free reign" instead of "free rein") and increasing use of AI to generate online media, even for 'traditional' outlets (e.g. a news article that used "site" instead of "cite").
I don't know what the solution is, but if we have increasing use of AI that is poorly educated to generate output to look forward to, the future will get into a vicious circle of 'incorrect/bad usage, people see this and think it's right, they use it themselves, the AI gets trained on the corpus of output from these people, it uses the incorrect/bad usage, people see this...'
Oh well, time to go and yell at those kids to get off my lawn!
This is obviously tied to a fear of enforcing rules, especially on poor (i.e. brown/black) students post-2020. Similar to the general decline in policing (in a general sense) and strong indicator (imho) of the usefulness of a more paternalistic approach.
If that is true, wouldn't you expect to see divergent effects by race? But the effect seems to be there regardless of race, or school demographics for that matter.
> Holder said the problem often stems from well intentioned “zero-tolerance” policies that too often inject the criminal justice system into the resolution of problems. Zero-tolerance policies, a tool that became popular in the 1990s, often spell out uniform and swift punishment for offenses such as truancy, smoking or carrying a weapon. Violators can lose classroom time or become saddled with a criminal record.
I gotta say -- my overall impression of zero tolerance policies has not historically been favorable... but also, is it not obvious that the things in this list are not the same??
Could we perhaps cut students a little bit of slack for skipping school, and try out some restorative justice or whatever, but still get the police involved if they are carrying weapons? This seems pretty obvious to me. And there are certainly a lot of anecdotal reports of "zero-tolerance" policies being applied to things like "fighting", in a way that gives severe consequences to both attackers and victims. So many of these implementation details just seem insane to me. I don't know what that says for the concept as a whole.
That's part of it, sure. There's an entire mosaic of different reasons why schools succeed or fail, and one element will be "disruptive pupils can't be disciplined effectively". If you have a bunch of kids who will be expelled from one school and bounce around from school to school, then there will be one final school that gets lumped with them (because education is a right and parents can sue if their kid isn't in school).
Since the kids have absolutely no interest in learning and probably have a raft of behavioural problems as well (future criminals in the making), plus the parents only care about "get them out from under our feet during the day" and not a genuine education, these schools are just holding pens until the kids are old enough to leave.
Well-meaning intentions from the top-down then imposed on the holding pen schools about "you can't expel them, you can't fail them, that's racist" means that they'll just be shuffled through the system, permitted to graduate, and dumped out on the streets because now they're somebody else's problem.
And the academic grifting movement about systemic racism declaring that "more black kids get punished in school than white kids, this is racism, so dismantle all disciplinary structure and expectations of academic achievement - because maths is racist too" just perpetrates the failure.
It's not the entire reason, but it's one more little tessera in the whole mosaic.
Put your predictor hats on! What do *you* think the 2025 or 2026 NAEP scores will be?
(If you want to look at the aggregate data it's at https://www.nationsreportcard.gov/data_tools.aspx . Do note that if you want *individual-level data* you will need to ask for a "restricted-use data license" though.)
I want to know what the hell happened to math in 2012/2013? Scores for most groups started plummeting after that. Was there a change in the tests? A change in the way math is taught?
I agree with the comments that the decline started in 2015, not with Covid.
Potential contributing factors;
1. Steady long term increase in non-white and non-Asian students in NAEP data. Other groups have significantly lower scores (per NAEP) and this is establishing a “headwind” for scores.
2. Movement of better students (often also white or Asian) out of public schools for obvious reasons. Another headwind. (I live in California and the number of responsible middle class parents that I know who leave their kids in public schools is really tiny)
3. Increase in single parent households.
4. Less reading, especially after 4th grade. More screen time and distractions.
5. Bad habits acquired during Covid by students, teachers, parents and schools.
6. Lowering standards in schools related to new teaching methods, diversity efforts and movement to pull down honor students for equity?
7. Movement away from phonics?
Even if all these are a little true, and some are indisputably true, the question still remains, not why scores started dropping after 2015, but more importantl, why did they go up for the 15 years prior? What changed in 2015 to no longer allow us to overcome these longer term headwinds? Does it have something to do with changes to Leave No Child Behind?
Said more concisely, we have had demographic headwinds for the last few generations, yet scores had been rising up until 2016. What changed?
My personal bias is to look at smartphones as an explaining factor. Smartphone unit sales peaked in 2015. The second derivative (rate of change in sales growth) turned negative in 2012 i.e. reaching saturation point. I would argue that any large social changes that occurred at/around that time interval should be checked for links to smartphone usage.
There is bound to be many reasons for this. One thing to consider is that zoom school wasn't a good fit for many students. For many kids the interaction with other students and the dynamic flow of events in the classroom are as important to their learning as the teacher's information dump. Zoom school was flat, like a can of soda left open overnight. It lacked the flow and interplay of classroom learning and, as a result, many kids lost their sense of connection and engagement with school. The world as they knew it ended, and school as they knew it lost it's luster. A lot of kids lost their sense of academic purpose and direction and it's not easy to reinvent that.
Or it might possibly be some of the best evidence yet in support of Sheldrake's theory of morphic resonance.
I think people not involved in education don't have a real idea of the work teachers do. It's not just information dumping, sometimes it is (metaphorically) spoon-feeding the pupils of lesser ability so that the teacher really is dragging them all the way over the line.
If you don't have that real-world interaction, a Zoom classroom is not going to work.
I used to think this in connection with the art teacher at my child's school. It was so physically demanding a job. She had them do stuff in several media, and in various "genres", year-round. The kids made the most beautiful and/or eye-catching products.* And they really did work on them, of course - but also it was plain to me that the teacher made sure that every last one was completed and decent-looking. There was much less of an effect where you could notice who the "best artists" were and who the worst.
I used to substitute teach and I rather dreaded "art" class because it was so much work!
I remember being in one such class all day and the teacher had deliberately left the easy assignment that the kids were merely to do some free sketching or brainstorming to ready their subjects for whatever the latest project was. A little girl came up to me and asked me to draw some animal for her. I had to confess I was no more proficient at drawing that animal than she would be.
"But you're an artist!" she said.
"No, I'm just a sub" I said.
"You have to be some kind of artist" she insisted, incredulous.
*It makes me laugh to think of my early grades spent in a Baptist church school. Their idea of art was e.g. to give us first graders boxes of toothpicks and glue. There you go. Make something. For a month when it was art time, we would drearily get our "tray" out and re-commence our toothpick work. A pile of toothpicks glued together. Then at Christmas we were asked to bring a phone book from home. We lived in a big city with an enormous phone book. Our art task was to fold each page of the book down. You read that right. After a week or two of this folding, our teacher opened the book out and spray painted it green as I recall - a Christmas tree. I don't remember any further decorating, perhaps we had folded too slowly and it was now past Christmas.
In your grade 4 reading graph, Oregon had the second biggest fall in NAEP scores, and I think that tended to be true across the different flavors of NAEP tests (reading vs. math, 4th vs. 8th grade). Oregon's public schools got extremely woke, perhaps even before 2020, so Oregon's performance appears to be, basically, a self-inflicted wound.
In general, it's worth remembering that the the pandemic coincided with the "racial reckoning" that began on May 25, 2020, which led to higher rates of homicides and traffic fatalities due to more lackadaisical policing. So, it's hard to disentangle the effects of covid and of the George Floyd racial reckoning on things like school grading standards.
In general, students from the upper half of society, those who don't get subsidized lunches, didn't seem to be hit all that hard by the events of 2020, judging from NAEP scores, just as their parents tended to perform their laptop jobs not too terribly during Work From Home.
But students from the the bottom half of society (those with free or cheaper lunches) got hammered on the NAEP, probably by being expected to study on Zoom. It's almost as if those kids tend to benefit from going daily to an organized place called a school where middle class grown-ups talk at them all day.
This is broadly the argument I make when the usual "school sucks and I hated it and it's prison for smart kids and I wish I had not been forced by the educational Gestapo to attend" conversation happens on here.
If you're smart, and if you have involved parents, then school probably doesn't make that big a difference to you.
And no, "now we can all do learning online so we don't need teachers and physical schools" is not a solution. Same applies: if you're smart and have involved parents who guide your learning so you're not slacking off or neglecting the subjects you don't care about, then online learning is fine.
If you're smart but poor/parents aren't involved (for a whole host of reasons, ranging from 'badly educated themselves so can't help even if they wanted' to 'you would in fact do better were you raised by wolves in the forest'), you may or may not be able to make up the difference.
If you're not Big Brain I Trounced The IQ Test kid, you probably need school discipline and structure. Throw in the worst of all worlds (poor kid, not academically in the highest range, uninvolved parents) and you are likely also getting free meals, guidance counselling and the host of support services that schools are now expected to provide - in effect, being quasi-parents not just educators.
There's only so much schools can do, when the kids go home and nobody gives a damn if they learned to read or not, and education is seen as some dumb imposition by society because it's never going to matter to you or anyone in your family.
And what Covid and the lockdown showed us is how the cracks really are there, and how kids can fall through them, and that no, not all kids are self-motivated little firecrackers who will be doing deep-diving on academic subjects if they are just allowed freedom and no mean old school forcing them to sit at a desk.
I think - as one who found school so intolerable, for reasons both of boredom but also my problematic temperament - I can without hypocrisy say that I believe we all would have flourished more in an environment that was closer to summer camp than to being uncomfortably ensconced for 12 years under flourescent lights in what was essentially indistinguishable from Class C office space, only worse.
Even a beautiful school would have made a difference to me.
I skipped school in late high school (decades ago) not because I was a bad student but because I was a good student. My school (in Toronto) was set up in such a way (terms instead of semesters, final exams only for mediocre and bad students, well-known grade fixing for better students in the humanities in OAC) that there was no reason to go to class every day after a certain point each spring. And the consequences were minimal, especially once I turned 18. Obviously mine was a very unique situation but there could be weird explanations for absenteeism that aren't "not good at school"/"don't like school."
From its begining NAEP has been a kind of Rorschach. Like many social indicators that's inherent in its design. You can offer more or less plausible hypotheses to explain the differences you see cross sectionally in a particular year or over time, based on associations with the "background" information NAEP gathers and reports on the students and schools sampled. And as the comments here demonatrate you are free to add whatever speculations you wish based on personal experience, ancecdote, and policy preference or bias. But you really can't make valid causal inferences based on NAEP data alone. The designers work very hard to try to make as many "construct irrelevant" variables, such as the test administration conditions, etc. etc. as "equal" as possible, but you never know for sure. Their official materials do caution you about the limitations, but I think the various "theys" are quite happy if they can provoke enough speculation to keep us paying attention.
Early NAEP didn't report in terms of performance scales - they just provided percent correct for specific items or clusters of items by various demographic categories. That was in part because of Tyler and Tukey (the original designers - an educator and an eminent statistician) -Tyler didn't believe in scale scores, and Tukey thought he could sqeeze all of the information he needed out of the item performance data. In those early days it was really very hard for anyone to make "What's it all mean?" sense out of the raw results, and it was in real danger of dying from loss of interest. NAEP didn't report in terms of psychometric scaling until an evaluation (one which I had some responsibility for, and now feel a bit guilty about) led to its being moved from the Education Commission of the States to ETS. ETS did its fancy thing and began reporting in terms of distributions of performance by students in various demographic and location categories on numerical scales with a standard range (like SATs) for the three age/grade levels of schooling - based on "matrix sampling" of students and content clusters. It is important to recognize that no student takes a full "NAEP Test" in any subject. The distributions of individual scores are distributions of "stat-rats," not real kids. They make a pretty rigorous case for why that particular fiction is o.k. for their purposes, but you really need to recognize that you are not looking at something like individual scores on an SAT, or for that matter on a State required accountability test, in most states -- so causal interpretation of the "individual" scores does get pretty "quirky" - which is a partial explanation for what Scott noticed as he looked deeper (?). Over its history NAEP found that it was still hard for the public to know whether they should care about the scale scores and changes in them over time, so NAGB (the governing board) proposed, and Congress allowed, on a "trial basis," reporting of scores in terms of "Achievement Levels" - i.e. "Below Basic, Basic,
Proficient, and Advanced" based on a rather loose eyeballing of the items that seemed to be associated with particular scale score levels by selected groups of Subject Matter "Experts," Educators, and Citizens (??) - so that NAEP basically could tell the public what was good, better, and not so good, and lo and behold, NAEP became "the Nation's Report Card." Those levels have never been sufficiently "validated" so Congress could be convinced to drop the "trial" monniker, and critics have often suspected that the levels are arbitrarily set high enough so that the public might always be encourgaged to view the results with alarm, and thus justify either sending more money from the Feds to public education or maybe abandon public education entirely - pick your
preferred conspiracy ..
My real point is -- you don't really know what these things measure or how much you should care about a 1 to 10 or 20 point change in any direction. They are consistent enough that they probably are measuring something, but how closely that something is related to something you care about and that you, or the schools, know what to do about, really requires a much closer look at actual curriculum and actual instruction in practice. And if you can do that, maybe you really don't need these slippery numbers at all - just get on with teaching and learning. Enough with the remote
High kids stay high, low kids stay low, this is the immutable and truly important point - students don't change places in the distribution relative to cohort
In addition to the ways that liberals are screwing over teaching a trend from conservatives is to reject any local tax increases, no matter how bad that is for the schools.
The plateau (and falloff) around 2007-2011 is of much more interest as far as I am concerned. I'm not sure what happened there because American public schools have been a steaming hot dumpster fire for a *much* longer time than that.
In many states, the standardized state assessment tests have been regularly (and purposefully) changed so that the decline isn't as noticeable. Obviously educrats understand the meaning of standardized in a very narrow, changeable way that allows them to massage data and claim improvement by changing the test whenever things start to look awkward.
You can get behind the paywall if you sign up for a subscription. That gives you a seven day grace period to poke around the archives. But if you don't cancel before the free trial runs out, you will be charged.
It could also be increased absenteeism because children are being infected repeatedly as schools refuse to integrate covid precautions such as cleaner air, and forcing students to remove masks, harming their immune (and other) systems, causing long-term sickness. Teachers are also increasingly off sick, which no doubt has an affect on the children's learning.
I see this very differently, and believe Covid is a key component. If you follow the copious medical research on the long-term post-viral impacts of Covid, along with rising death rates and sickness of people, you see that it’s not a ‘culture of absententeeism’ that is to blame but that people are actually sick a lot more. Add Covid health damage (including cognitive damage) to this prison-like environment and high stakes testing requirement of our schools, plus overloading teachers, and we have an education system that is collapsing.
Anecdata: I second the hypothesis that lowered standards (forgiveness and equity justified) were sticky post covid - but they were also accelerations of existing trends, not a sudden cliff out of the blue.
The once fashionable Education Reform movement (which thought George W. Bush's criticism of "the soft bigotry of low expectations" was the height of wisdom) favored by billionaires burned out in the mid 2010s as the impossibility of much narrowing the racial gap in school performance sank in, and was replaced by wokeness and its emphasis on how racial gaps are due to white evilness and that whites therefore must pay. Not surprisingly, wokeness discovered a way to lower the racial gap in flunking students: don't flunk any students.
The irony is that it seems that it wasn't actually impossible, at least not at a practical level (maybe politically impossible). See for example https://www.weforum.org/stories/2015/04/what-makes-a-school-successful-2/ (this is from before Fryer was cancelled).
Out of the loop - why was he canceled? Because discipline consequences were naturally going to fall on poorer and minorities more and so were inherently racist or something?
Because he was the guy his social class chose to prove that cops are racist, and he failed to do so.
No mention of the possibility that Long Covid just lowered the collective intelligence a bunch?
I guess we'll have to see children who weren't even born when Covid hits. Unless the long covid hits even that far
COVID is now endemic. All the kids will be exposed.
Yes, but is it still the same COVID? My impression was that the wildtype and beta variants were particularly nasty, unusal pathogens, wreaking havoc in unexpected places, while the variants that replaced them were, while ridiculously infectious, much more standard respiratory viruses with effects more in line with a regular cold or flu.
I think milder symptoms over time are mostly the effect of immune response (from both vaccination and natural exposure), not milder variants.
I think it's both. There's extensive evidence that COVID evolved to be more contagious and less acutely harmful.
That was my impression as of Christmas 2021: suddenly, everybody has come down with this new Omicron variant, but, remarkably, people aren't dying in the numbers you'd expect.
Thank you, evolution.
able to share any of this? I have not read extensive evidence of this.
I got omicron while unvaccinated and as a newbie to covid. It was but a mild cold.
Are you sure you were a newbie to covid? You might have had an asymptomatic infection before.
In fact, weren't most cases of covid at all times rather asymptomatic?
Right, for example the Delta variant from India in mid 2021 was nasty, but the Omicron variant that arrived in America in late 2021 was highly infectious, but not as lethal.
My impression is that basically we got lucky in that covid evolved in South Africa in later 2021 to be less ferocious but more infectious, so by early 2022 after Omicron faded, we were largely over the hump with covid: most people had by now either already been infected or vaccinated or both, and the subsequent variants weren't as bad as pre-Omicron ones.
But I don't hear many authorities saying that anymore when it comes to this fifth anniversary.
Maybe there is some deterministic reason why it had to evolve that way, but diseases don't always: e.g., smallpox stayed vicious for thousands of years.
I just tried to do a bit of research on smallpox to check your statement. Apparently, there's no proper consensus on whether it got milder or not.
So I guess, if it got milder, it didn't get milder enough.
For eg syphilis we know it got milder, and milder enough that there is a consensus.
Thanks.
Other respiratory ills have run wild this year. I work with the elderly, but I suspect the same with kids. Lots of sick kids since covid?
This seems unlikely to me, but probably worth examining to be sure.
Quick googling suggests that surveys find something like 6-8% of adults report themself as ever having had Long Covid. (A smaller fraction currently have it, but this many report a period of several months where it affected them.) I think that numbers are generally lower in children, but it's not completely out of the question if it's higher.
If 6% of students had Long Covid, and this caused just a minor dip in their test scores, this probably couldn't explain such big overall test score dips, so it would have to cause big drops for the students who were affected, if it's going to cause this.
That would mean that most of the students with Long Covid would probably show up in the "lower performing students" group, and not the "higher performing students" group - the "higher performing students" would have their test scores drop a little bit, as 6% of their number moved out and are replaced by the top of the "average students" but the "lower performing students" group would probably drop more significantly, as 6% of their number moved down, and many of the rest of them are replaced by people from higher groups that had dropped to below the old "lower performing students" group.
That *might* be compatible with some of the data, but it seems unlikely to me.
I don't think it's plausible that 25%+ of young children have long COVID, unless we're completely misunderstanding the situation. Also, if that were it, why wouldn't high-performers be affected?
cognitive reserve
It doesn't have to be the kind of full on long covid that people report, which stops you working or at least gives you noticeable symptoms. A small amount of circulatory and/or brain damage would presumably have similar results in knocking the performance of marginal cases down a bit, and wouldn't necessarily be noticeable?
A _lot_ of my friends report reduced energy levels that never quite fully recover after their first tangle with covid, not to the level they'd go to the doctor about it, but noticeable for those who pay attention.
I find it extremely plausible that lots of 80 year olds came down with severe covid, eventually recovered, but found that their cognitive abilities had taken a sharper downward turn than simple aging would cause. Getting old can be like the Hemingway character who went broke in two ways: gradually, and then all at once. A major illness can cause a sizable downturn that you never recover from. For example, if you look at pictures of my mother before and after her cancer at age 61, right before she looks middle-aged and right after she looks elderly, and she stayed elderly.
In contrast, I find it less plausible that lots of grade school kids were damaged long term by covid. Maybe, but kids tend to pop back from all sorts of vicious ailments.
You say that like the high-performers weren't affected, but the data you found on that question was decidedly mixed.
There are recent studies showing that covid can subtly decrease cognitive function without patients even noticing. There are tons of studies that show covid can cause changes in the brain in even mild cases. I do not think that self described rates of Long Covid are accurate.
https://www.nature.com/articles/s41586-022-04569-5
https://doi.org/10.1056/nejmoa2311330
These are just two studies out of dozens in this vein. I am not a scientist or researcher, but I do think that they're on to something based on my anecdotal experiences.
Judging by self-report, long covid seems like an ailment that afflicts the better educated, such as New York Times subscribers, more. But the biggest drop in NAEP/TIMSS scores has been among the lower half of society, the free lunch kids.
The better educated are more able to advocate for themselves and will therefore identify unclear symptoms as long covid rather than just feeling a bit crap. Meanwhile small neurological defects will affect people who didn't have as much margin to deal with problems more.
COVID seems to hit older people the hardest, and younger people less so.
At least in terms of acute effects, which may not be correlated with subtle long-term effects.
I guess in the sense that dead people don't have "subtle long-term effects".
What? Of course dead people don't have subtle long-term effects. But those who had a mild form of the illness, even one that went unnoticed, certainly can. We know, for example, that adults 50+ who experienced a mild case have significantly increased risk of heart attack. There may well be other effects that we don't yet know about.
Adults 50+ are the type of people I suggested were hit harder by COVID, rather than school-age children.
That's certainly true, but doesn't rule out long-term and as yet unidentified effects on school-age children.
Show any data which supports this wildly speculative hypothesis.
The "wildly speculative" hypothesis that there may be long term effects that aren't correlated with the observed severity of the illness? I gave one example, the significant increase in the risk of heart attacks. I don't know that there are others, but we don't know that there aren't.
I meant more generally the notion that some sizeable portion of children have long Covid. AFAIK the thing that correlates most highly with long Covid is trait neuroticism.
Long COVID is a specific-ish thing, and I'm not saying I believe lots of kids have it. I'm saying it's possible there are as-yet unidentified long-term effects.
Well, one might start looking at this by looking at other countries with same Covid levels but different policies. Would be useful as a comparison to US NAEP scores, as well.
Wild speculation, but maybe it affected teaching staff especially, and this could partially explain the lower scores? "1 in 5 educators who, according to a recent EdWeek survey, have experienced the emerging, mysterious illness known as long COVID." in this article https://www.edweek.org/leadership/1-in-5-educators-say-theyve-experienced-long-covid/2022/04. This 2023 paper from the UK (https://pmc.ncbi.nlm.nih.gov/articles/PMC7615205/) seems to say (I'm not sure I understand the "likelihood (ORs)" stuff) that people in teaching have 11.6% higher chance of getting long covid, and that 22% of people with long covid in teaching report a reduced function by "a lot". The raw prevalence of long covid symptoms in "teaching and education" slowly rose from ~2.5/3% up to ~5% (Figure 2). I'm not sure what's the impact of ~1% of the people working in education having a reduced function by a lot (22% of 5%, roughly).
Teachers were enthusiastic early adopters of working from home over Zoom.
Long Covid has always been around; we used to call it hypochondria.
Exactly - there’s no such thing.
People just blaming something other than themselves for various ailments.
Sequela from viruses is not a fake or made up condition. Some people get it from the flu, and there are reports of people getting it from the original SARS outbreak in the 2000s. There is a possibility that some people are making it up, but long covid is well studied enough at this point that to dismiss it as psychological is ignorant.
There are quite a few people who barely survived covid -- e.g., I can recall reading an article about an ER surgeon who spent a month in intensive care on one of those heart-lung machines reserved for the most desperate cases. Amazingly, he lived (ER surgeons tend to normally enjoy rude good health), but I doubt if he'll ever be quite the same vigorous man he once was. If that guy said he was a victim of long covid because he could no longer handle working in ER, I'd say, "Yes, sir."
But long covid enthusiasts tend to argue _against_ there being a correlation between how severe their case was and how severe their subsequent debility is.
Perhaps they are on to something. Or perhaps they are hypochondriacs. Or perhaps some are the former and some are the latter.
Yes, assuming long covid exists, hypochondriacs also still exist and presumably in similar proportions in the overall population as before Covid.
That would require evidence and a plausible epidemiology. Long Covid is rare, especially among children. It would also show up in IQ tests and AFAIK there hasn't been anything unusual happening there. SAT scores actually went up slightly post-Covid.
Teachers I know got burned out/fed up and quit [during and after covid]. I haven't seen any data to know if that is a widespread phenomenon, but if you're looking for why schools could be worse now, that's where I'd start looking.
This was my thought too, as a current teacher. The profession is much easier to get into now than it was 10 years ago, due to widespread teacher burnout. The loss of experienced math and English teachers could have a big role to play (and is potentially testable).
A lot of Baby Boomers retired during covid as their 401Ks did well.
Everybody complains about Boomers, but you'll probably miss our competence when we're gone.
Mostly, we'll miss the part where you were earning and investing money, rather than cashing it out to finance retirement. And really, each of you is going to miss the part where all the rest of you were earning and investing, when you each try to retire and cash out.
Luckily the rest of the world is still getting richer and willing to buy the stocks and bonds that current American baby boomer retirees are selling.
The rest of the world is also still earning and investing.
Of course, eg China is rapidly getting older, but that'll hit too late for the baby boomers to worry about.
I'm not finding a lot of data to corroborate either (apparently teacher turnover went from 20% / year to 30% / year, with high poverty districts affected to an even greater degree), but my money is on teachers as well. I bet a lot of teachers changed district (eg relocating closer to family), changed methods (eg remote teaching), or left the profession altogether during COVID, including very experienced, top performing ones, and the void they left hasn't been filled yet, because the new blood hasn't ramped up and acquiring teaching experience is slow. Simple data like "average years of teaching" for 2014-2024 could answer that, but I'm not finding it.
>I'm not finding a lot of data to corroborate either (apparently teacher turnover went from 20% / year to 30% / year,
Going to 20% to 30% seems huge to me!
First of all, it my only be a 10% change in absolute numbers, but it's a 50% increase! Most systems cannot change a sudden 50% increase in volume of some major fundamental challenge they face.
But more importantly, the fact that it is a yearly rate means it is cumulative, having exponential effects on seniority.
If we model the 20/30% chance/year as hitting teachers at random (which is wrong of course, but illustrates the point), then at 20% you get 32% of teachers making it to 5 years, and 11% making it to 10 years. At 30%, you get 17% making it to 5 years, and 3% making it to 10 years!
So it feels like that 10% absolute change would be plenty to massively decrease seniority among teachers, which we could reasonably expect to have a big effect on education quality.
It does depend on where the dropout is concentrated. If you had a lump of retirement, and then any new teachers drop out much more often, you could get the larger portion of teachers who have been there for a while staying there, plus a churn of new ones.
New teachers haven't got their lesson plans set up yet, etc.
Why would this be a new phenomenon? Have teaching circumstances changed in the last few years?
Teachers tend to be white because there isn't much affirmative action in hiring public school teachers due to a 1990s federal law promoting good teaching. On the other hand, there is huge affirmative action among principals and administrators. Hence, there has been a quiet racial cold war going on in the public schools between the teachers and the administrators.
So, during the Great Awokening, especially during the Racial Reckoning that began May 25, 2020, it hasn't been unusual for white teachers to retire. Stocks are high, so retirement accounts aren't paltry. But are retiring teachers recommending to their kids that they go into teaching?
This is also my standing hypothesis. No data (and no faith in teacher quality rankings anyway), but lots of teachers in my extended family and they all say there loss of many of the best teachers has been intense post-covid.
Even more subjective (though it should be possible to study) but I think covid coincided with (and partially drove) a new level of politization of public views of schooling, more intense if unhelpful fights about curriculum, etc. I would expect all of those things to degrade schools over time.
Anecdotally it's the same here; those close to retirement are taking it earlier and those not close are fleeing the public schools, either for non-teacher jobs or to teach at private schools.
Stated reasons all seem to have a theme of reduced autonomy in the job (the two biggest reasons I have heard are more stringent curriculum requirements and inability to discipline problem students, but there were other minor complaints that my brain filed as "administration trying to control teachers").
Unlikely. Teacher quality has very little impact on educational outcomes (less than 5% of total variance, IIRC).
Really? Do you have a citation handy? Compare top google result: https://www.epi.org/publication/books_teacher_quality_execsum_intro/#:~:text=Their%20research%20identifies%20teacher%20quality,important%20predictor%20is%20teacher%20quality.
I know I saw a review article in the last year that put the figure at 5-10% but I can't seem to find it. This is what ChatGPT said:
Chetty, Friedman, & Rockoff (2014) – Their study found that high-quality teachers significantly impact long-term outcomes, such as college attendance and lifetime earnings. However, teacher quality explained about 10-15% of the variance in short-term student test scores.
And frankly I'm skeptical of 10-15. Generally I would expect this topic to be a nexus for motivated reasoning. The political incentives lean strongly towards valorizing teachers and minimizing the role of genetics, so I would expect the published literature to overestimate actual teacher value.
Here's a review that puts it at 9%: https://www.sciencedirect.com/science/article/pii/S0883035523000630
Teaching From Home over Zoom sounds easier than teaching in the classroom.
It's easier in that you can pretend everyone is out there listening and taking notes and thinking about what you are saying. But they're not, and when test score come back, you go WTF!!! How am I going to get them engaged?!?
It's phones
Are we going to get NAEP results in 2026? I thought it was on the chopping block along with the rest of the Dept of Ed.
'Thanks to my administration, our students no longer have plummeting standardized test scores!' (because we stopped measuring them)
Why use resources to measure a problem we lack the will to fix ?
Because if you can show that it is staying bad for long enough, or that it's getting even worse, the will to fix it might emerge.
Reasonable in the abstract. However the instruments have said broken for decades. At some point doing the same thing is madness.
But the will to fix it might involve taking away power from the people who don't want to fix it, which is further motivation for them not to measure it.
We spend a lot of money and attention on this topic as a society, and most politicians at least talk about their plans for it.
Look at teh first graph in this post, before the current drop-off these scores were steadily increasing for decades. We absolutely *do* do things about this in general, we're just facing a current acute problem that people are trying to figure out.
The people doing the measurement don't have to be the people to fix it.
Specifically, the government doesn't have to fix all the problems it measures.
And it is impossible to fix this problem as long as there is a teacher's union to protect incompetence and make sure bad, ignorant teachers are never, can never be fired.
States have done that off and on without the en/discouragement of an interfering DOE. Constantly changing targets allow you to get away with a lot.
My entire career originated with Congress allocating in the famous 1964 Civil Rights Act the sum of one ... million ... dollars to James S. Coleman to demonstrate quantitatively that the reason for the racial gap in test scores was due to less spending on black than white students.
When looking at education statistics one needs to keep a few things in mind:
a) The population changes over time. E.g. California was ~87% white in 1970 or so. Today California public school classrooms are over 50% Hispanic. One can hope/pretend/assume that those two populations score equivalently ... or not.
b) The population changes over time. How has private and homeschooling percentages changed over time. If more of the top scoring kids aren't being tested then we'd expect the test scores to drop.
c) The tests sometimes change where they put cut lines.
Yeah, but look at the graphs. It's not a gradual change since 1970. It's going up until 2015, going slightly down until COVID, then (on at least some of the tests) plummeting over COVID, then going a bit back up later. Percent white doesn't look like that at all!
Let me generalize: The raw data is cooked :-(
A (medical) doctor named John Jacob Cannell looked into state education test scores in the 1980s and discovered that all 50 states were above average. He spent some time investigating and wrote a short booklet title "The 'Lake Wobegon' Report: How Public Educators Cheat on Standardized Achievement Tests."
It generated a tiny bit of news coverage but no change.
He followed it up 20 years later with this:
https://files.eric.ed.gov/fulltext/EJ874474.pdf
My working assumption for most/all US public education testing data is that it is badl flawed. On purpose.
On top of that we also have sampling problems and scoring problems. But the raw data should be considered suspect (in either direction) until proven otherwise.
Do we think the data became significantly more or less cooked in the last few years? If not, I'm not sure how much explanatory power it has for the current topic. But I'm looking forward to reading the article at the link - thanks for sharing.
If capitation grants depend on results, there's a strong incentive there to cook the data. I don't know if that's so, but it seems plausible. Nobody wants to attend or to fund School A where the results seem bad (because they are poor, because the returns are honest) so it's in the interest of School A to nudge the worst performers out the door (or at least prevent them from taking the tests) and fudge the remainder to improve the appearance of things, simply for survival.
The NAEP results don't look terribly faked. They mostly look like what most of the other data validates.
The most dubious NAEP bias strikes me as Texas's somewhat high scores. Texas appears to take efforts to encourage it's dimmest 5% of students to call in sick on NAEP testing day. But it's pretty marginal.
There is currently a "teacher test cheating ring" scandal in which a not-small number of teachers paid someone to take the test that allows you to become a teacher in Texas. The locus was Houston but every few weeks uncovers a few more teachers in distant parts of the state, who paid someone to take the test for them - up to about 500 now. I don't know if this is a hard or an easy test. But either way, it suggests a culture unlikely to regard testing integrity as a goal.
My basic point is/was that the data is unreliable. In both directions.
Partially because the folks reporting it collectively don't want it to be reliable.
I *could* dig around and try to figure out what is going on. But that is work and feels a lot like forensic accounting where you try to figure out where the losses have been buried.
I don't find that interesting here so I don't feel a need to explain the results. Since I don't trust the data they don't matter.
NOTE: I got to play this game about 15 years ago in a non-education field when GPUs were *just* starting to become popular for high performance computing. Lots of papers got written with 20x, 50x, 100x speedups over CPUs [the record was 3,600x speedup over a CPU solution]. When my group ported some code we found ... 4x at best.
What was going on was that there were a number of ways to sandbag the baseline and different groups used different mixes of those ways to determine how much faster their GPU code was than their CPU code. They made their CPU code slow to show a great speedup.
I'll further note that the claims were coming from university research groups and being presented at conferences. This wasn't marketing nonsense from Nvida.
When I dug into this 20+ years ago for US education statistics I found similar games (though different strategies). The SAT was made more coachable a number of years ago, for example. And the SAT taking population changes based on things like states requiring students to take it to graduate.
Education achievement statistics are full of this sort of nonsense.
For the GPUs I no longer feel any need to explain weird results. Either the folks reporting the results explain what they did and why they got the result and how it makes sense or they don't. But I'm not interested in trying to reverse engineer.
Finally, 15 years ago when I eventually got an Nvidia support engineer on the phone to discuss my 4x we walked through what was going on and he said, "Yeah, 4x is about right." The other groups *were* scamming the numbers. And Nvidia knew it as well as the HPC research community as a whole. No one cared.
It's tiresome for people to assume that nobody has ever looked into test scores before, so therefore they must be obviously "cooked."
The most notorious thumb on the scale of NAEP scores is that you have to have 90% attendance on the day of the test, and Texas tends to have 91% attendance, while other states often have 95 or 96% attendance. It looks like Texas's high NAEP scores relative to its demographics (e.g., compared to ethnically similar California) is in part due to Texas encouraging its 5% dimmest students to take a day off when the NAEP comes to town. But, that's also not a really huge deal.
In general, the NAEP is administered by professionals who want to do a good, fair job.
I would say in my experience in 35 years in education reform, the answer is no. It has been being cooked for decades.
This is a high-value comment. Thanks for this info, it's super interesting. Kinda makes you wonder how much of the country is caught up in pantomimes like this in order to fool the people who control the money. I suspect some large fraction of government- or charity-sourced funding is being directed by shell games like this.
It just illustrates why maintaining a profit motive in all things is so important. Everything immediately becomes dishonest in its absence.
The NAEP was significantly upgraded around 1989.
Based on the first graph it kinda looks like California is getting smarter over time, with a minor drop recently, so Hispanics are clearly smarter.
I agree that point a) makes little sense (or perhaps means that more Hispanic students has contributed to a steady increase in NAEP scores in California until ~2017?), but point b) might have merit: during COVID, tons of families who could afford it moved their kids to private schools (not because they were good schools, but because they were open), and these kids aren't taking NAEP scores. If these kids used to pull the average up (and it's plausible that parental income and IQ are correlated), that mix shift might explain the drop.
Is there a principled reason why the first graph starts at 1998 (when NAEP started in 1969) and has a score range of 200-225 instead of 0-225?
No. I’m glad you mentioned it. We should teach basic statistics in secondary school (that’s high school to you) even for kids with little math ability just to be aware of statistical tricks. Zooming in is one of them.
There’s actually very little change in that graph. It’s largely steady
https://flowingdata.com/projects/dishonest-charts/
You might appreciate this.
I always hear 'we should teach [important thing X] in school'. As if people remember or retained much of all the other thing and skills they are supposedly taught in school already.
Illegal immigration massively spiked in 2021 with the Biden administration, which would look on a time series similar to the aftermath of covid. Given the younger demographic profile of migrants (whose kids might be in school even if the parents are undocumented) and the shrinking pool of native born children, adding millions of young families in the span of just a couple years could have an outsize effect, especially when you consider that (apart from their own test scores) an influx of non-native speakers likely has disruptive effects on everyone else's learning. And IIRC, the most common finding on school quality is that ongoing class disruption can bring everyone's learning down much more efficiently than widespread good behavior can bring anyone's learning up.
You can look at NAEP scores by race/ethnicity.
The events of 2020 -- covid and George Floyd -- were massive cultural changes.
Distinguishing between the effects of lockdowns and the racial reckoning, however, is challenging.
"a) The population changes over time. E.g. California was ~87% white in 1970 or so. Today California public school classrooms are over 50% Hispanic. One can hope/pretend/assume that those two populations score equivalently ... or not."
Unlikely this has anything to do with a decline over 6-10 years.
"How has private and homeschooling percentages changed over time. If more of the top scoring kids aren't being tested then we'd expect the test scores to drop."
They probably didn't get ignificantly more private schools running over the last 6 years that siphoned off good performers, so more homeschooling taking the best performers out of circulation would be the only real explanation offered here.
Statistically, homeschool did massively spike, and it was almost all the parents well-positioned to help their kid, start a "pod", etc. Anecdotally, I have several friends who took their high-performing kids to homeschool during covid and just never went back.
I know of two groups of parents who started a "pod" with a professional teacher, and they kept it going post-lockdown because everybody involved preferred the situation and could afford it. They both stopped when the kids were secondary school age.
Also, the local Catholic schools can be a good bell-weather for parents' perception of the public schools, since they are typically the most affordable private option. E.g. when our local school district eliminated honors classes, the enrollments of non-Christians went up, and there was a similar bump when they reopened sooner than public schools.
The NAEP reports scores by ethnicity. https://www.nationsreportcard.gov/reading/nation/groups/?grade=12
In my state the "standardized" state test changed three times in 5 years. Makes it really hard to compare data or know what is actually going on. Not that it matters. Bad schools seldom face consequences that can make any difference for the kids.
What about test scores from students who started school after lockdowns ended?
The post mentions a couple times that in 2026 when students who started after covid reach 4th grade and start taking these tests it'll provide more information one way or another.
I would not expect a sudden reversion to pre-COVID scores - I have a child in that cohort and their first-grade teacher described that year as "half the class is basically feral, and I'm doing triage to get everyone to grade level." That will show up even in the high-performance kids, because they're not getting the attention they would normally get in a class because they're doing well enough. I think you won't see kids truly unaffected by COVID until the 2028 or 2030 numbers.
As for the pre-COVID drops, I'm surprised phones weren't mentioned as a culprit. Would explain the larger drop in the 8th grade cohort vs. the 4th.
Like with a lot of stuff, I think Covid accelerated trends that were already happening. It seems pretty clear a big chunk of the population really doesn’t think of school as a priority and wants teachers to take on 100% of the responsibility for teaching their kids, with zero help from parents. I also suspect that fentanyl (and other drugs) took a decent number of parents and caregivers out of the picture. And of course screens are a huge part of it.
I think the analysis about missing a year of school assumes a reasonably intelligent kid with parents who are involved . I suspect the impact is much, much greater with a kid on the left side of the bell curve who doesn’t seem the inside of a school for a year and loses the habit of attending, along with most of his friends.
> Like with a lot of stuff, I think Covid accelerated trends that were already happening.
Yes, exactly. Test scores were already in decline due to systemic and/or cultural problems; COVID just time-warped the trend forward by about a decade; and my prediction is that generative AI will warp us forward just as quickly (if not faster):
https://thewalrus.ca/i-used-to-teach-students-now-i-catch-chatgpt-cheats
> fentanyl
I think you’re overestimating the prevalence. It was surprisingly hard to find data, since almost every source only trumpets the overdose rate. But I’m seeing estimates below 0.5%. Is that really enough to compete with more common factors like “remote learning” or “every teacher who can afford it quitting”?
https://www.sciencedirect.com/science/article/pii/S0749379724001053
Teachers I know say that behavior and mental health (already declining) eroded markedly and never returned to pre-Covid levels, creating challenging learning environments. That has pushed teachers to leave the profession. 50% leave within 5 years of starting and veterans are seeking early retirement like never before.
The behavior is out of this world. Its a tough situation with a lot of causes and everyone is suffering.
Antisocial behavior has increased throughout society since the pandemic, no reason to expect schools to be exempt I suppose.
I've heard the same from multiple teachers. Behavior issues got significantly worse and disruptive students are frequently causing issues leading to "room clear"s.
They complain that administration is just not helping much and teachers are directly having to deal with extreme and violent behaviors.
The argument I hear, but haven't researched, is that teacher shortages might be to blame, caused by poor pay and poor treatment (by admin and by students; I wouldn't be surprised if political and cultural polarization is causing kids to be more chaotic and obstinate).
Anecdotally, I live in a pretty good school district in a very good school state, and teacher turnover has been awful. My kid's school barely has enough teachers to function, lacks needed specialists, and is facing a significant budget shortfall next year. A high school in a nearby town has some classes that don't have teachers; the kids just go to the room and...hang out, I guess? I suppose the diligent ones probably work on homework. But this isn't a poor town, they just can't find teachers.
Has teacher pay gotten substantially worse? Have staff shortages? (My gut sense is that the answers are “no” and “yes”)
Teacher pay has been blamed for every negative metric for decades, which is not to take a position on whether their pay is sufficient, merely to say that I don’t think that is something that changed in 2020.
I think teacher pay hasn't gotten absolutely worse, but pay in other jobs has gotten significantly better since the pandemic (there were a couple years of significant labor shortages).
Similar to baumols cost disease. Teachers as educated people can earn more elsewhere more easily
Not particularly like Baumol's Cost Disease but quite a lot like teachers wages not keeping up with wage inflation.
Wage inflation is almost exactly what Baumol's cost disease is.
Specifically wage inflation in sectors that haven't seen a productivity increase, as a sort-of spill-over from sectors where productivity has increased.
Baumol's cost disease is specifically about productivity enhancement in other fields. I have not seen anyone claiming that the pandemic caused a bunch of fields to experience productivity enhancement. Pandemic (and post-pandemic) wage inflation is generally attributed to retirements and employment reshuffling where employers had to offer increased wages to fill vacancies. One could certainly argue that Baumol's contributed to changes in the quality of teachers over the last 30 years but that is somewhat removed from the comment to which you were responding, "...teacher pay hasn't gotten absolutely worse, but... since the pandemic."
Right.
That would be my suspicion as well, though the question of pay is interesting, as teacher pay as the unions fight for it (i.e. more pay and greater job security) probably doesn't help much, but if the best teachers truly are leaving for less stressful jobs with better pay, we want some way to get them back. If it's not pay it needs to be about stress/chaos. Is that an administration problem or a broader cultural one? Probably a bit of both, but either way it's unlikely to be an easy fix.
So the most elegant solution might be to tie higher pay to performance. But how do you measure performance? It's not as simple as many other jobs. It can't be raw test scores as that would just funnel the best teachers to the richest school districts. Test score improvements could work, but I imagine there are diminishing returns over time as scores enter and move past the fat part of the bell curve. Reputation? Peer accolades? That could incentivize inter-departmental conflict in a field that should be extremely collaborative. Reviews from students/teachers? Could pressure teachers to never fail anyone.
I don't think there's an easy answer. I suspect it comes down to the messy cultural stuff. But if a core issue is that many of the best teachers are leaving, some way of bringing them back might help push back against decline. People who have actually researched this will know more, I'm sure.
> So the most elegant solution might be [...]
The most elegant solution is to let private organisation figure out how to attract good teacher, or otherwise how to provide good education for their customers.
Just like we let private organisation figure out how to attract good staff to server their customers in most other parts of society.
(I'm deliberately not expressing an opinion over whether this is a _good_ idea. Only that it would obviously be the most elegant solution, by virtue of removing a special case.)
If the pay is the same but the job is worse (misbehaving children, misbehaving parents, misbehaving administrators...), then it follows that people would quit.
I think that the work / pay ratio has become significantly skewed. Teachers are expected to enforce tons of policies meant to rectify society and family failures or weaknesses, like mental health programming, behavioral programming, free menstrual products, addressing drug use in the classroom, being a mentor to kids without good parents, etc., as well as tons of special interests, like mainstreaming delayed children. They work every night and every weekend, for pay that isn't keeping up with the increased workload.
Also, they are just plain burning out.
If it's the teachers, that could have been exacerbated by Covid. I have a friend or two in the school system and they were pretty unhappy about how their administration handled the pandemic, enough that from what I heard lots of people who could retire or swap jobs did.
From my (mostly high school) perspective, chronic absenteeism looks likely to be a big cause. Administration has been cracking down heavily this year, in a way that makes me think it’s been much worse than usual in the last 3-4 (this also matches anecdotes from veteran teachers).
[My school is kind of a weird one though (“public magnet”, ish), so take these ideas with grains of salt.]
We’re also attached to a more conventional middle school (a fairly poor-performing one in a fairly poor area). And I hear stories from teachers who spend more time with the middle schoolers. There’s one teacher’s experience in particular I want to relate:
Mr. H was a literature teacher for a few years around a decade ago. Seven years ago, he left to do a PhD, and then he came back last year (as Dr. H). He had both middle and high school classes, and related to me a number of stories of total disarray resulting from poor standards and insufficient administrator attention.
Dr. H was particularly concerned with attendance among the high schoolers—he expressed in no unclear terms that absenteeism had become much more common since his previous tenure. And he told me that the middle schoolers were total agents of chaos—fighting in the halls, cursing administrators and teachers out, and facing almost no consequences. One or two day suspensions at most. He made it through a year and a half, then earlier this year, Dr. H left our school.
This is all anecdotal, but is pointing to two main issues from my view (and I think some other commenters are saying similar things): lower standards (academic, attendance, and, I think crucially, behavioral), and good teachers leaving. Even those that weren’t burnt out by Covid directly (like Dr. H) were driven away by the total structural failures that it created.
The real lesson here is to never, ever, ever get an English PhD. You'll just end up back teaching middle school.
> and facing almost no consequences.
This is the key part.
It is not possible to handle large groups of people, if there is nothing you can do about those who choose to be disruptive -- because, statistically, there will always be someone like that.
On internet, you have bans. In real life, you have the police; people who break the rules can get fined, or put in jail. Voluntary associations can kick out misbehaving members.
At school, in the past we had corporal punishment, which is no longer legal... and that is good in general... but the problem is that we have no replacement. Or rather, all suggested replacements have some serious problem. The results are... predictable. (Although apparently the people who make the rules are unable to predict even the most obvious things.)
The replacement is kicking the student out of class every time they begin to misbehave. Chinese schools no longer have corporal punishment either, but misbehaving students simply don't exist in Chinese classrooms because they know there's zero tolerance for that kind of behavior from the teacher, the principal, the parents, the classmates, or anyone else.
That would be nice.
But it requires support from the school. As a teacher you can't simply send the misbehaving student away (they would probably misbehave in the hall, and potentially hurt themselves, and then their parents would sue you). You need someone you can call to take the student away, and some room where those students are collected. So this must be official school policy.
And even then, I am not sure whether it is legally okay (this depends on specific country's laws) to kick a student out of too many classes. At some moment, the parents could argue that the student is deprived of mandatory education. So you need the entire system to say that yes, this is perfectly legal.
So while the solution exists in theory (and in China in practice), no teacher can implement it alone, and maybe not even a single school.
I don't think anyone really has a satisfying explanation of what's going on. Even weirder is there are assessments of adult skills that show worse performance over the same period of time. (https://www.wsj.com/us-news/america-us-math-proficiency-falling-1b5ac73c) During this period there's been a shift to more online assessment, but test-makers typically check for that sort of platform influence so it doesn't seem like a legit explanation.
Did we all become dumber due to catching Covid? What happened with all those studies that showed long term damage to grey matter, etc? Unlikely to be the explanation, but still somewhat relevant
> Did we all become dumber due to catching Covid?
Yes.
No one read those studies, too long and had some really tough vocab in them.
The decline started with the post-trurth era. Nothing matters, only soundbites from short vertical videos matter!
It's absolutely ridiculous how many commenters here seems to have the obvious psychocultural shift in their blindspot. (Scott included.)
Kids also are famously not living under a rock, they see what's going on in the world, see AI is coming, wars starting, climate changing, and ... who the fuck wants to study for tests nowadays!?
This is a theory. Ideally it would be useful to check whether it is supported by data. Tik Tok user base did in fact grow by a factor 10 between 2018 and 2022, so the timing seems right. This is a starting point but it's a far cry from proving a causal connection
COVID causes brain damage.
Sure I remember the papers claiming that, but lots of stuff causes brain damage. This does not always affect academic performance in a measurable way.
You're correct, but there have been several recent studies showing decreases in IQ from covid infections.
I can't see the article. What specific test are they talking about?
I wonder if short-form video has contributed to this. I imagine someone has already analyzed this sort of data stratifying by short-form video consumption. That would make sense, since apparently this worsening hasn't happened among high performers, who we could suppose might consume less short-form video.
If it's only going in in the US, that's a clue in itself
I feel like you didn't account for what deeply addicting all of society to tech has done, and what it then has done to parents.
The rise of autism (and autism like symptoms in neurotypical children) fueled by technology addiction is madness!
I am inclined to agree with you, but I haven’t made a deep study of it. It is my intuition that it is true.
As a serious tech addict I think this is the true cause. Covid exacerbated things because we all got incredibly hooked during it.
Yeah, good habits take a lot of time to build, and covid broke some of them.
This, my theory is that smartphone and social media addiction are the primary drivers of decline, and COVID resulted in an acceleration of that trend due to everything going online and people being forced to stay inside all day. Even after everything reopened the increased tech dependency remained. You can also see the same trends appearing in assessments of youth mental health, which itself also affects educational ability.
Just my two cents that this is an excellent point. Covid, or at least the response to it, was an accelerant for all sorts of negative trends.
Mistaking correlation for causation would be madness.
The climb in autism diagnoses preceded smartphones. It definitely preceded modern social media.
I believe pocket technology has rewarded short attention spans. That is *not* the same thing as ASD.
I think you could trace TV addiction more broadly towards autism also. It's only autism if it's detrimental, otherwise it's sparkling neuroplasty 😜
how does autism rise because of technology? People are born autistic are they not? They could not contract autism, or am I woefully mistaken?
Autism is diagnosable if it leads to adverse effects. Likely more people than assumed has the genes for Autism, but I would argue that many of those people find coping mechanisms and never get diagnosed. I would say that technology becomes a special interest for many folks, neurotypical and diverse, but imo technology accelerates neurodivergent people towards Autism.
It has little, if anything to do with covid. As you observed, the decline started before the pandemic.
The real reasons in my opinion:
1. Educators for years have been shifting to teaching strategies that don't work (e.g. "whole language" learning, "inquiry-based" or "number sense" math, virtual elimination of science and social studies for elementary and middle school) because traditional methods are boring for teachers. My girlfriend is an elementary school teaching assistant. Her school switched back to teaching phonics this year and younger kids reading scores are skyrocketing. The kids beyond about 3rd grade are still struggling, probably because they missed a developmental window for learning to read. It's sad.
2. Elimination of physical textbooks. My kid is expected to reference a text and write a response on an 8" tablet. After accounting for the virtual keyboard the text and the word document are relegated to two tiny ~2x3" squares.
3. Increased absenteeism is partly becuase a) Everyone is sick all the time now and b) Because during those years of "virtual learning" parents were finally able to observe how terrible and inefficient modern classrooms are. Once you realize your kid's school is bullshit, your drive to make them attend school is diminished.
4. Probably the phones.
Re: 3, I think mores changed a lot here. When I was a kid, you went to school unless you had fever or were vomiting, staying home because of anything less than that was malingering. That clearly changed a lot during COVID, I don't think it totally returned to what it was.
I hate technology in the classroom! It needs to go! In some places they had out tablets to kindergartners.
Or at least, if you insist on technology, buy large screens.
The current situation is worse than both extremes: we want technology because it is cool, but we buy crappy technology because a good one is more expensive.
Writing on screens is a bad idea. I could imagine reading from screens, but they would need to have the same size as the paper textbooks. I checked some e-shops, the readers of that size cost about thousand €. (I imagine that this is partially because few people buy them -- if the educational system made a huge order of a few hundred thousand pieces, that could probably drive the prices down.)
Rereading my comment, I kinda conflated 2 things above. This is what I get for writing fast and on my phone. For item 1: teachers abandoned phonics and rote math instruction because it's boring and because of modern teaching fads. Science and history/social studies are deemphasized in elementary and middle school education to make more time to teach to standardized tests and because of some weird theory that young children don't absorb concrete facts well. To the extent you do see science or social studies taught in younger grades it tends to be vague stuff like "what is a community?" rather than explicit facts.
I also think science hard or soft are sacrificed because we need to focus on fundamentals because the faddish methods failed, so obviously we need to try them harder and longer, right?
> some weird theory that young children don't absorb concrete facts well
This goes completely against everything I know about how people learn. Concrete facts are easy; abstractions are difficult if you don't know the concrete facts.
But yeah, the fads in education usually go both against the science and the common sense.
Right. My girlfriend is a teaching assistant. When she has the rare opportunity to teach facts the kids absolutely love it.
Maybe the decline partly owes to growth in the population of students who are illegal immigrants. They don't speak English and received poor educations in their home countries before coming here. Average them into the standardized test scores and they drag it down.
Do all the legal ones speak English? I’m just asking.
No, not even close. But more importantly: rates of English proficiency have been *rising* (https://www.pewresearch.org/short-reads/2024/09/27/key-findings-about-us-immigrants/sr_24-09-26_immigrantupdate_3/) overall, and mostly stable among enrolled students (https://nces.ed.gov/programs/digest/d23/tables/dt23_204.20.asp) - there was a slight increase in 2021, where the data cuts off, but that's a: well after the decline in scores starts, and b: nowhere near large enough to explain the observed changes, no matter how poorly they perform.
This doesn't seem particularly likely as an explanation, seeing that this effect seems to be pretty similar in nearly all states, while immigration (both legal and illegal) is concentrated in some states and nearly non-existent in others. West Virginia, Montana, Mississippi, South Dakota, and Maine are the five states where less than 5% of schoolchildren are from immigrant households, and they are all over the chart with some having the worst declines in recent years while others having relatively low declines.
Similarly, the six states where children from immigrant households make up more than 30% of schoolchildren are California, New York, New Jersey, Florida, Texas, and Nevada. These states are all right in the middle of the pack.
https://cis.org/Report/Mapping-Impact-Immigration-Public-Schools
If you think that the distribution of immigrant children in the past few years is different enough from the distribution in 2021 to cause some significant errors in this comparison, I'd be glad to learn of some relevant data.
Do the numbers for this match up in *any* sort of a remotely plausible fashion? Have you checked?
My recollection is that numbers of undocumented immigrants in the U.S. has remained pretty steady for the past decade and change, in the range of 12 million or so[1]. Not only is it missing the sort of sharp increase that would be necessary for your argument to hold, but it's just plain to small anyway: a demographic that makes up 3% of the total U.S. population isn't going to have such a dramatic impact on test scores. Not, it's possible that the proportion of school aged kids within the demographic is larger than the percentage of the population, and has grown relatively larger in recent years. But we'd be talking about a BIG demographic difference. Where's the evidence for that?
[1] Though of course "total number remains steady" is not the same as "all the same people are in the group." Some die or leave, others arrive.
"numbers of undocumented immigrants in the U.S. has remained pretty steady for the past decade and change, in the range of 12 million or so[1]."
Since we know that large numbers are crossing the border into the US every year, this would require equally large numbers leaving every year. This is not happening. So the number is not true.
It's especially not true since that *same* number (actually "11 million") was being quoted back during the Reagan amnesty thing forty years ago.
Yes, I'm old enough that I was around and interested then. I do remember things. No, it was not believable then and it is definitely not believable now.
"My recollection is that numbers of undocumented immigrants in the U.S. has remained pretty steady for the past decade and change, in the range of 12 million or so"
What do you mean? During the Biden Administration, the news was constantly full of reports of high levels of illegal immigration. The statistics in this report bear that out: https://www.migrationpolicy.org/article/biden-immigration-legacy
And an increasing share of those illegal immigrants are now minors: "Beginning about ten years ago, the pattern changed and unaccompanied minors and migrant families from northern Central America accounted for a pronounced share. This trend has sharply increased since 2021, as many irregularly arriving migrants have come from elsewhere in the Western Hemisphere and farther afield, often seeking asylum in the United States (see Figure 2). "
"a demographic that makes up 3% of the total U.S. population"
The share could be significantly higher than that. No one knows for sure.
https://insights.som.yale.edu/insights/yale-study-finds-twice-as-many-undocumented-immigrants-as-previous-estimates
"The share could be significantly higher than that. No one knows for sure. "
And yet, it *still doesn't matter.* It could be 4%, 5%, 6% or even 8%--hugely more than the number I stated--and it still wouldn't come REMOTELY close to carrying the weight of the above argument. Look at the graphs again. They're not driven by an 8% total share of the student population. They're ESPECIALLY not driven by an INCREASE in a demographic that (at very, very generous estimate) tops out at 8%. Like, suppose population of school-aged, undocumented immigrants in the U.S. literally *doubled* from 2019 to 2024, from 5% to 10%[1], you'd still have to argue that an absolute change of 5% in the total student population was responsible for those massive changes in the graphs.
So unless you're going to try to sit here tell me that somehow 30-40% of the student-age population is undocumented immigrants with poor English skills (across every state, somehow), the argument above is bunk. At best it's misunderstanding driven by poor statistical acumen. At worst it's an attempt to shoehorn your hobby horse into an unrelated topic.
[1]Where we are generously adding an extra 25% for demographic skew.
If that were true, you’d expect correlation by state and especially county, because the illegal population is not evenly distributed.
I don't know what proficiency means but I hear a lot of people around me using a hybrid of English and Spanish. Less an insertion of words into a given sentence, though obviously there is some of that especially with newer or vocational-related terminology, than a flowing back and forth within the same conversation, perhaps reverting to the one or the other when speaking more quickly - whichever is a little faster and more comfortable. I think this is unconscious. Now, I know that being "bilingual" is considered a praiseworthy sign of genius and virtue - but my impression of this hybrid is that it is a cruder version of both languages. I enjoy learning new words in Spanish, for instance, and Spanish speakers naturally as any of us would, enjoy being appealed to in the role of master; but I often find that the person I'm asking does not know the word any more than I do.
This may be the way you make the US Mexican, and is not unexpected; but it probably isn't making for better reading literacy.
My newest Spanish addition is halcon, for hawk, which is obviously delightfully close to falcon and so easy to remember.
My guess is that it's a combination of schools' COVID response and longer-term factors involving standards, acting in synergy.
I would hazard an even stronger guess that it has almost nothing to do with the physical effects of COVID infection.
the high-performers have inexplicably great years from 2017 - 2019, but then return to average?
——————————————————-
Agreed that the data is all wonky, but this one at least strikes me as having an obvious answer: high-performing students are much more likely to have parents who are paying attention, and a lot of those were aggressively home-schooling during the lockdowns. Then those kids went back to the public schools. Oops.
I interpreted that part as "nerds are doing better when there are no bullies around".
Bullies may or may not disproportionately target high-performing students (though my money is on not; the relevant feature of "nerdiness" here is social isolation, not academic ability) but it would need to be *absurdly* disproportionate to produce such a large effect on high performers and be undetectable otherwise.
This feels right to me.
I home schooled my kids during Covid and my older son got through 4 grades of math in one year. My kids got more teaching during the pandemic, not less.
I question the premise that chronic absenteeism is due to not bothering to show up. I think it's due at least in part to more illness. Covid-19 as a thing you can get sick with has been added to all the other things you can get sick with. And chronic absenteeism is missing just two or more days per month according to that AEI report, which could easily happen with one febrile illness.
I don't think the existence of COVID doubles the amount of time that the average child is ill.
The existence of COVID lowers the standard for what counts as serious illness enough that it probably DOES double what it was before.
You've nailed it. The school where my spouse works always had the policy "if your child has had a fever in the last x hours, keep them home." But they were very lax about giving that message to parents, checking temperatures at school when kids seemed sick, calling parents before the end of the day to pick up their feverish children, etc.
Since the school reopened post-COVID onset, they've been very diligent about promoting and enforcing that policy. A kid who has a minor fever on Saturday evening but is fine on Sunday morning is expected not to be at school Monday.
COVID can and does cause long-term immune damage. Hardly implausible that people are actually sicker more. Anecdotally, I have definitely observed that.
"people are sicker more" - sure, that's plausible. But twice as sick *on average* is an enormous effect, especially when you consider that that average includes the vast majority of people never report any long covid symptoms at all.
Does that happen a lot in children? Does it happen often enough to make them sick twice as often as before, in numbers large enough to affect the statistics?
One illness might make two days extra of absenteeism. But I would be very surprised if there's a significant number of children that average one illness of this level of significance *per month*. There would have to have been a lot of children who were just one month short of this threshold for one additional annual illness to have pushed them over this threshold!
I don't think you have to be chronically absent every month to count in the statistic. Any kid that has missed two days is chronically absent for that month. You can be back in compliance the next month.
Edit: the AEI definition is annual, 2+ monthly absences increased from 15% to 28%. The NAEP definition is monthly, 11% to 22% but for 5+ missed days. Bit hard to make sense of, actually.
What about the emergence of WFH? That makes it much easier to let kids take sick days.
This is likely part of the effect. If both parents had to be in the workplace, absence of the child due to illness was probably serious. One of the parents has to take time off.
If the child can be at home because one of the parents is there, then the criteria for the parent pulling the child is probably lower.
If there's a hybrid arrangement and "the kid can listen in on the app" the criteria likely gets even looser. Although I suppose that counts as "present", so maybe that's not a factor.
No idea if there's more two-parent families now; that may be another factor.
Ah that’s interesting! So if the average child has Covid once a year, and is counted as “chronically absent” in the month they have Covid, then we should expect “rate of chronic absenteeism” to go up by about 8% post covid compared to before. (Maybe more if COVID infections are concentrated in January, February, and March, compared to summer or winter break; maybe less if a significant number of kids go more than a year between covid infections that are significant enough to take them out of school for three days.)
Anyway, thats about half of the observed rise
That seems like a really odd measure for "chronic absenteeism". It doesn't measure what people think it does.
OTOH I can see this being approved by a committee as a compromise metric. "Compromise" being defined as "stupider than any of the originally proposed metrics".
I looked at the scores at: https://www.nationsreportcard.gov/reading/nation/groups/?grade=4. Interestingly, there is a uniform decrease across most categories, by race, by school type, by region etc. The only exception seem to be the Asians, who have seen their scores increase. Not sure what to make of this information.
This is anecdotal but I felt like during the Obama years the dominant view among Asians I knew is that we’re all going to assimilate into a post-racial mass and it’s okay to accept regression to the mean because we’ll be fine, as expressed by this article pretty well: https://www.nytimes.com/2018/06/22/opinion/sunday/asian-american-tiger-parents.html
After Trump and especially after the anti-Asian COVID hate the feeling is more that regression to the mean is dangerous since we won’t really be securely accepted as equals and so we have to still push our kids to be the best they can be so they have in-demand skills to take care of themselves. The author of that above article himself was recently blocked from a judgeship despite having pretty strong qualifications, and I think the views he expressed are clearly minority views among Asians today.
Maybe there is evidence that relative Asian test scores started increasing faster in recent years than before.
"especially after the anti-Asian COVID hate the feeling is more that regression to the mean is dangerous since we won’t really be securely accepted as equals and so we have to still push our kids to be the best they can be so they have in-demand skills to take care of themselves."
That time period correlates with Asians becoming more likely to vote Republican.
Makes sense considering who was perpetrating the majority of anti-Asian violence and who was benefitting the most from anti-Asian discrimination in hiring and admissions.
The graph informs you that the increase in Asian scores is not statistically significant.
From my experience with kindergarten and 2nd grader in a "good" district, the standards and expectations were lowered somewhat in last couple years, likely a DEI effect.
I suspect it's more likely a "dealing with the post-covid chaos" effect than a DEI effect, especially if it's just in the past few years. DEI was quite significant in the 2010s as well, if not more so!
It really ramped up to a much greater degree after the 2020 riots, though, so that's consistent with a slow decline accelerating after 2020.
One thing I didn't mention on the post but just thought of:
Lots of concerned parents didn't like lockdowns and put their kids in private/home school. Presumably these parents are selected for some kind of positive train (caring about education, having resources) so their kids probably do better than average. If many of these better-than-average kids stayed in private/home school after the lockdown, that lowers averages.
Does NAEP make any attempt to correct for this?
Assuming the first Google search results are to be trusted (which they might not be) homeschooling rates in the U.S. are only somewhere in the 3-6% range. It seems very unlikely that removing ~3% of students from the dataset would have this kind of noticeable impact, even you picked specifically the top 3% (which of course homeschooled kids won't be).
I think school performance also went crazy in countries like Germany where homeschooling is not allowed.
Home schooling is basically a white thing. An outflux to home schools wouldn't cause the decline in black and Hispanic scores.
If not NAEP, how about PISA?
1. It would be great to see comparisons to other countries. Not seeing them is sadly a very common occurrence on blogs by US-bloggers, even otherwise very good / rigorous ones like Scott.
They would help disprove hypotheses like "this is because of wokeness" or "something something republicans making minorities uncomfortable at school something something" (both unlikely, but likely to be brought up in online arguments because culture wars are a thing), as well as stuff like "the tests changed"/ "now it's harder to cheat" / "students use computers instead of paper and that's bad" / "some overlooked legal change removed teacher incentives to teach to the test" (a lot more believable).
2. How come nobody is mentioning smartphones? There's plenty of scientific evidence at least pointing in this direction, and there seems to be a notable uprise in short video / Tiktok use that coincides with this timeframe.
For 4th Graders?
According to Common Sense Media, 40% of 10 year olds have a smartphone. See page 22 here: https://www.commonsensemedia.org/sites/default/files/research/report/8-18-census-integrated-report-final-web_0.pdf
See my comment elsewhere in the thread for a comparison to Australian NAPLAN data; the tldr is that the dip doesn't exist in Australia, so covid-related theories are mostly off the table (and US-specific culture war theories are still firmly on it, so go nuts).
This also suggests smartphone- or social media- related theories are off the table.
Didn't Australia have a *very* different pandemic than we did, though? It seems at least plausible that the lowering of behavioral and academic expectations that anecdotally seems to have happened here wouldn't have happened there because our cultures interacted so differently with COVID.
Yes! I was internally screaming this while reading the article - if you want to compare the effects of different policies and disentangle that from culture etc, there's a whole world out here with a lot more variation than just the 50 US states.
Made even worse by the final chart, from TIMSS - an international test covering 50+ countries each round. You have to make a conscious decision to ignore the international comparisons and just pull out the US data.
Excellent point. Trying to decipher trends by looking at only one country's data is like trying to understand why chess moves were made when only looking at one color's pieces. When AI gets slightly better, I'm going to have my AI assistant remind me to check international comparisons every time I see a discussion of trends in the US.
In the meantime, I had Deep Research look into it. Preliminary results (i.e., me not confirming its output) suggest the negative trend is international. It called the drop in OECD countries’ PISA scores from 2018 to 2022 “unprecedented” and "scores revealed that many countries… fell back to levels from a decade ago."
“In short, math proficiency declines have been broad and substantial, affecting both advanced economies and developing countries. No region was entirely spared, though the severity varied.” Reading scores also declined, but not by as much as math. This is a consistent pattern (speculated to be because reading is easier to do independently and remotely).
As another commenter mentioned, it also notes Australia is an exception (along with a few other countries).
“In summary, the decline in U.S. test scores is not a uniquely American phenomenon.”
It also blamed the pandemic and the pandemic responses (unprompted). It says “countries with prolonged school closures tended to suffer larger learning losses…. A global study of TIMSS 2023 data confirms a strong association: nations that kept schools closed for only a few weeks generally stayed closer to their pre-pandemic achievement trends, whereas those that closed for many months fell far behind.”
It also notes some differing opinions, such as OECD director of education Andreas Schleicher, who said, "COVID probably played some role but I would not overrate it. There are underlying structural factors, and they are much more likely to be permanent features of our education systems that policymakers should really take seriously." I did confirm this person exists and the quote is real.
Googling the quotes the AI gave you, I found this source (not written by AI, as far as I can tell): https://www.weforum.org/stories/2023/12/oecd-pisa-results-maths-reading-skills-education/
"The Programme for International Student Assessment (better known as PISA) 2022 saw an "unprecedented drop in performance" across the OECD regions. Compared to the last edition, in 2018, the mean performance in reading fell by 10 score points, while maths fell by nearly 15.
This drop in maths was particularly apparent in countries like Germany, Iceland, the Netherlands, Norway and Poland, says the OECD, which all saw drops of 25 score points or more."
The fact that scores have not only not recovered, but continued to fall is particularly striking. I would naively assume that the rate of change for learning is higher if your learning has been previously slowed (it’s easier to catch up than to learn at your age appropriate rate). But even if the learning rate wasn’t higher, I certainly wouldn’t expect it to be worse. Yet if the scores are still falling, then that suggests the negative impact of the COVID years has compounded in later non-COVID years such that today’s 4th graders now are worse off than those who had just exited COVID in 2022. To me, that does suggest something structural happening (or I have a very incorrect mental model of learning).
It makes sense to me that learning has critical periods that are hard to recover from, both neurologically and in terms of standardized curriculum.
IE, if you hit the 'bad' patch of schooling at an age where you are supposed to be learning the fundamentals of literacy or reading comprehension or mental arithmetic or etc., then it makes sense to me that this is very hard to recover from later.
Especially so if the teacher is forced to follow a curriculum that assumes you have already mastered those skills, instead of noticing your deficiency and changing the plan to focus on correcting it.
I think many people and even your initial post hint that schools will work hard to catch up for any learning loss they see, and that there is both a lot of slack (in the wastefulness of a school day) and enough institutional drive (incentivized by test score monitoring) to make sure we get things back to where they were initially. But remember that everyone is health sensitive now. So any signs of sickness kids are much more likely to be sent home, and so any claims of (real or feigned) sickness also result in being home.
What on Earth happened in Louisiana public education?
Isn't New Orleans 100% charters post-Katrina? They might have totally different trends.
Yeah, looking at the state graph, I'm less interested in whatever happened during covid and more interested in whatever Louisiana is doing right.
Louisiana was sitting near the bottom of education ranking/results and instituted a number of reforms just last year. These reforms resulted in them going from 43rd to 32nd place.
My guess would be the teacher training reforms. As of 2021 Louisiana requires elementary school administrators and K-3 teachers to complete "a foundational literacy skills instruction course that is based on the science of reading ... and includes information on instructing students regarding phonemic awareness, phonics, fluency, vocabulary, and comprehension." Mississippi did something similar a few years prior, and saw significant improvements as well.
And of course it also helps that they were both performing very poorly to begin with.
Maybe the best readers from all other states have moved there?
I don't know if this has any explanatory power, but I'd like to see a version of this analysis where, rather than comparing a raw difference in score 2019-2024 by state, you look at a deviation from the trend--are there certain states that had been declining on average, pre-COVID, and then others that had been steadily improving?
I blame the proliferation of short form content, and primarily TikTok.
I don't know how many middle-class suburban American families you all know, but of my extended family who fits the bill, a concerningly high proportion of the male children are addicted to (what is essentially) softcore short-form porn, and generally attention-killing content. I know this has been an ongoing problem and accusation agains the internet more broadly, but I think this time it really is "different."
I really don't know what effect this would have on grades (which I assume correlate to my made up metric: "attention span"), but I can't imagine anything good. The drop in grades approximates the proliferation of TikTok, Instagram Reels and Youtube Shorts.
And as long as we're blaming new technologies, turning in assignments written by AI is, as I understand it, completely rampant and nearly impossible for the average teacher to combat.
'Lowering standards' is one explanation for why grades are staying high while standardized test scores are falling.
But 'all homework is done by AI and gets As, everyone flunks in-person tests because they were using AI instead of learning' also accounts for the same trend.
AI writing (at least the kind I run into online, or generate) is not at all good at disguising its voice, and it's not at all rare for a forum post to be greeted with "fuck off chatGPT"
Are elementary/middle school aged kids using the free models really good enough to disguise that voice but not good enough to just answer the questions correctly?
The transgressive thrill of cheating is not to be underestimated. People may get a sense of satisfaction from fooling their teachers that they might not get from merely studying the material.
LLMs have very distinctive voices, but weak human writers generally *don't* - one bullshitting high schooler sounds pretty much like the next. If you're not already attuned to GPT-voice, as I suspect most people are not, it probably sounds like a bullshitting high schooler too - especially when given high school essay prompts. The entire student body mastering English grammar all at once probably *should* set off some alarm bells, but if you're not already looking for it, would you notice? I doubt it.
I believe this is an example of confirmation bias. We tend to notice when someone uses AI because of its distinctive "helpful assistant" tone, but we often overlook its more nuanced use as a supplementary tool—rather than simply asking it to "write a whole book report."
accusing a student of cheating is very serious, implicitly the standard is "beyond a reasonable doubt". watermarking and AI detectors don't actually work. so it's usually pretty obvious but also not conclusive enough to throw the book at someone.
especially as even honest kids may have their writing style influenced by LLM outputs that they read.
According to my sister, detecting AI writing is extremely easy, because her middle school English students will open up ChatGPT *in front of her.*
Well, remember Scott's test of ability to tell AI from human in terms of generated images. Some people were good at the task, but 'good' at the 80% range, not 99.9%.
If you are a teacher deciding whether to give a student a zero on their assignment and refer them for disciplinary action because you think they used AI, are you ok with being wrong and doing that to an innocent kid who did their own work 20% of the time? The way parents complain these days, I think you'd get fired pretty fast if you insisted on that policy with that accuracy rate.
NAEP scores show similar declines for boys and girls:
https://www.nationsreportcard.gov/reading/nation/groups/?grade=4
What's going on in Louisiana??
Didn't a lot of veteran teachers retire during Covid? Aren't class sizes larger now?
Also, where is the reading/doing-math-at-grade-level line on those graphs?
How much does the stuff kids do in school matter, as compared to the stuff kids do at home? Anecdotally, as a college student, I've found that my outside-the-classroom activities have way more of an effect on my reading/writing abilities than anything I do in the classroom. Getting outside, seeing friends, not spending too much time on my phone, etc-- these things all have an effect on my mental concentration and performance that is 10x the effect of any specific assignment I do in the classroom. If I e.g. spend 10 hours on social media on a Sunday, then can't concentrate the next day, that has nothing to do with my teacher.
After the pandemic, kids have been vastly more online than they used to be. If a kid is going home from school and playing video games for 8 hours, no amount of fancy classroom management or test prep can counteract that. My take is that kids' home lives have changed (most importantly: more time on smartphones), and that's what accounts for declining test scores.
In 2020-21, I worked at ETS as a user experience consultant on the NAEP. The major problem we had to solve was to maintain long-term trend while transitioning from a device-specific digital assessment (i.e., the NAEP was being administered on locked-down Surface Pros that the DOE shipped to school districts) to a device agnostic digital assessment (i.e., one that could load in any browser).
I left for a full time role in tech before this work completed. But I'd like to point out the NAEP timeline here: https://nces.ed.gov/nationsreportcard/about/timeline.aspx
Psychometricians are supposed to be norming the test based on usability changes that might affect scores. Digitization poses a lot of variables that can confound long-term trend. I wrote an internal report about the effects the "digital divide" has no test takers that need to use assistive technologies. Basically, low-SES kids can perform even lower on digital assessments than they would on paper-pencil because they, for instance, may not recognize the icon that opens the calculator tool as they are less familiar with digital cues.
Anyway, I don't have much data to share but I thought I'd point out in 2024, the device agnostic assessment launched on Chromebooks. For all we know, the test wasn't normed very well against long-term trend and could have usability issues affecting it. As the timeline shows, the test was paper and pencil until 2017 (for math, reading and science), which is when a lot of these oddities you noted started happening.
Thanks! This is a valuable insight.
I could imagine that part of the effect is due to differences in the test from one year to the next. Particularly because a score decrease from 216 to 214 seems small, in both relative and absolute terms. It appears larger in these graphs where the y axis starts at 200.
On the other hand, other tests in other countries show similar trends.
A lot of cases where there seem to be sudden dramatic changes end up being a measurement issue. Any chance that's what happening here?
Check out Conor's post earlier for a potential explanation
one interesting result. Maine and New Hampshire are right next to each other. Maine is the worst. New Hampshire is on of best. This could be a cause of investigation. Why do two schools just over the State Line have wildly different experiences?
"Absenteeism" is also happening when school is just not running classes when they're supposed to. Ever since COVID, my local school district has a number of "asynchronous learning days" on its calendar. These count toward the state-mandated number of school days for the year... but they're a lie. They are 100% days off from school. Maybe some kids are doing some homework or something, but there is no class happening in any way any different than a holiday.
Even rigorous schools in my area have become very comfortable with calling off school in a way that was unheard of 10 years ago. There's a private school nearby that had a reputation for being open unless the door physically wouldn't open due to snow drifts, but since 2020 it's fallen in line with other schools. This year it closed due to a forecast of mild snowfall that didn't even materialize.
A really obvious way to check the COVID-related hypothesis, especially, would be to compare to data outside of the U.S. Obviously you won't be able to compare test scores 1-1, but it should be easy to see if a given country's benchmarks-of-choice show the same sort of sudden drop. If it's much less common outside the U.S., that points to something specific about the U.S. education environment. If it common among many countries, that points strongly to COVID. In the latter case, asking which countries were least effective and what they do/did differently may prove fruitful.
One would need to be careful to control for differences in vaccination rates, infection incidence (for example, differences in mask-wearing and ventilation), etc. I don't know how possible it is to control for this stuff, other than interventional (or quasi-interventional) studies done within individuals, which doesn't really give as much insight into population level trends.
That would be necessary in order to draw specific, robust conclusions, sure. But the point here is mostly to do a quick sanity-check on the main hypothesis (and perhaps to locate plausible alternatives if it fails). The thing Scott's looking to explain is a very large, sudden drop in test scores. If there is no corresponding large, sudden drop in test scores in a significant fraction of broadly-similar countries, the answer *could* be [complicated statistical reasons that made it show up across all U.S. states but hardly anywhere else]. But it's *more likely* to be "this hypothesis is wrong and there's some alternate cause instead." The signal in the U.S. is very strong. It's possible but very unlikely that the same cause could apply to many other countries, but get masked by noise in almost all of them despite being strong in the U.S.
Speaking as a math tutor, I've seen dramatic effects from school closures. Anecdotal, I know, but apparently the research backs me up.
In fact, for some of my students just skipping school entirely might've been better than online classes were. At least then they wouldn't have been taught to cheat.
Looking at similar tests in similar countries could help eliminate some possibilities. The NAPLAN tests in Australia are equivalent; if it's Covid-related then there should be a similar dip:
https://readwritethinklearn.com/blog/naplan-2023-results/
There's no covid dip. (There is a dip in 2023, but this is attributed to the test being moved two months earlier in the school year.) So I'm inclined to eliminate covid-related theories, whether due to the lockdown or the disease.
Probably rules out "it's the phones" as well unless AU has very different norms around kids and phones than the US.
having been to both australian and american schools in elementary, middle, and high school (albeit decades ago of course), I'd say that there's really different norms between them in general: way more private / religious schools in Aus; all have uniforms; way more sex-seperated schools (I went to an all girls high school); school sizes / class sizes much smaller; schools start an hour later, much more emphasis on rote learning; middle schools dont exist (high schools are 7-12, primary school k-6); compulsory schooling to year 10 only; trade school after year 10 are normalized; college major choice is restricted/determined by end of high school exams scores. In terms of phones, Australian schools were much freer about discipline and restriction--there was no hesitation to take phones or send people out of class. The whole system is radically different in my recall, but I'd be curious to hear from someone who went more recently.
Were schools closed in Australia as long as they were in the United States? Quick AI search seems to suggest they weren't.
Very state-dependent. Western Australia had one of the shortest and least disruptive lockdowns in the world (they eradicated the virus in a month and then went back to life as normal) while Victoria had one of the worst (because they were intent on eradicating the virus but kept failing). Melbourne had a total of 263 days in lockdown, supposedly more than any other city in the world, although I'm not sure how many days the schools were closed.
Anyway, the two biggest states have about half the population of the country and they both had extensive lockdowns so if lockdowns were a big issue you'd expect to see it in the NAPLAN results.
Also you'd expect to see a strong state-by-state effect where Victorian students got a lot worse and Western Australian students didn't. But we don't see this:
https://www.acara.edu.au/reporting/national-report-on-schooling-in-australia/naplan-national-results
Australia barely locked down aside from Victoria, and as you say the scores *did* drop in 2023 (for confounded reasons). The by-state data you've linked only starts in 2023, so we can't see a time series for how Victoria *changed* relative to WA or SA (the least-locked-down states).
Also, additional confounder not mentioned - NAPLAN testing was absolutely FUBAR'd by technical problems the last few years. Stuff like students seeing the test, not being able to submit results then re-taking the test a few days later when they've had days to think about the questions. The transition from paper tests to online tests happened over several years from 2018-2022.
I don't think any useful conclusion can be drawn from this data.
I remember reading your earlier article and thinking "what applies to one child missing school is not likely to apply when all the children miss school together" -- there is probably a swimming-with-the-current effect when you jump back into a peer group that's made progress while you were away. There's pressure, and help, to get back to your (known and obvious) place within the group.
I think the vibes argument also applies: though it's merely anecdotes, there were a lot of news articles in post-COVID educational journals about how students seemed much more checked-out, stressed, disruptive, and distractible after the pandemic. You could imagine the pandemic -- both its immediate effects and the secondary knock-ons, ranging from increased mistrust of schools, angry woke teachers, angry anti-woke parents, etc -- diminished the overall level of motivation in schools, both for teachers and students, in a way that is really hard to quantify because just applies a small negative effect to everything across the board. The absenteeism issues might just be one small manifestation of a much larger "school bad" vibe.
Also: notable that the "best" year in most plots is ~2012-2013; that was the last year where <50% of Americans owned smartphones.
you’re overthinking it. If you were a parent during this period you saw the devastating impact that the Covid shutdowns had on schools and kids. It wasn’t just missing some classes it was a cultural shift. As some people said below it was in some ways a continuation of existing trends - but it took all the worst existing trends and supercharged them
If that's true, how come states that tried to keep schools open aren't overperforming?
I suspect a combination of lower standards and worse social behavior standards. I have a son-in-law who quit high school English teaching a few years before Covid hit - and from what I have heard, his peers who stayed teaching have found it far more frustrating and challenging than before.
But the point I want to raise it that the schools tend to think that they are teaching their students and that changes in their curriculium and methods are responsible for changes in the learning levels. And this may be true to some degree, but highly educated / professional parents are at least as well educated as the teachers and are likely to have higher standards and expectations for their children. And they are highly likely to supplement their children's education with their own guidance or paid supplementation. In either case, the education of these children diverges from that of their peers who are not having their education supplemented.
I spent hours flashcarding math facts into my kids - and reading to and with them until they mastered reading and starting reading for knowledge and enjoyment - which meant very limited screen access. I also supplemented their math education - mean dad, bad dad. But they mastered the material.
I can easily believe that the Covid wave forced more parents to be active in their kids education and some parents in good disctricts may have slacked off - doing all that education work takes a lot fo time and effort.
But kids who rely solely upon school for their education are going to be badly impacted by lower standards, worse social behavior, and addicting screens - short videos and the social network social games are far more addictive and diverting.
My kids hated my intrusiveness when I was overseeing their education. But they are following the same pattern with their children, and discuss approaches with me.
I have a couple good friends who are or were teachers and it does seem like the teacher burnout from the profession becoming increasingly regimented to state curricula with little autonomy and the general rise in antisocial behavior among kids and harsh discipline enacted to counteract it, such as the regular lockdowns that happen at a middle school one friend is at, accounts for most of this.
One thing to look at could be private vs. public schools. Many private schools went through the same “wokeness” that public schools did, some even banning advanced classes and tests, but generally also give teachers a lot more autonomy so they aren’t burned out and have smaller classes and softer disciplinary policies. Did student performance in private schools also drop or only public ones?
The Common Core curriculum was implemented about 2014-2015.
What do NAEP scores show about the persistence of changes in performance? Is there any clear relationship between 4th grade scores and the scores of the same cohort in 8th & 12th grades?
Something we must not forget is the long-term general trend of grade inflation. We've all come to take it for granted. But there are no grades higher than A, so this inflation cannot go on forever. I know an admittedly small number of children, so my sample size is small. But the kids I know are "smart" kids in the sense that they can figure stuff out, can crack clever jokes, and have a lot of common sense. They are curious, adventurous, and great at solving puzzles in video games. But even by age 9 or 10 they can't tie their shoes, spell out their entire name, give you their address or phone number, or multiply two single-digit numbers together. And the kids I do know, attend American public schools and an Australian private school, so it's an international problem. What's going on? I think it's just that parents don't do any educating at home. They shrug and send their kids off to school. Schools do nothing, kids are bored, administrators and teachers all get a paycheck, everyone goes home at the end of the day, and it starts all over the next day. At the end of the semester they get report cards with all A's. Surely this can't continue through college? Oh, but it does. As long as they don't major in STEM, they'll be fine. They graduate, fail to launch, and blame "capitalism" for their failures.
I am a long time reader of Scott Alexander. This is my first comment ever. I hope he sees it. :)
The No Child Left Behind Act included both requirements that schools measure student performance in various subgroups (such as low income students) and strong incentives for school districts to raise student achievement. Failure is often rewarded in education policy—such as more funding for low performing schools. But NCLB actually had economic incentives for schools to perform well; schools that were failure were required to pay for tutoring by outside entities. So student performance improved both because of the tutoring AND the incentive schools had to avoid paying for it by increasing their test scores.
When congress ended the No Child Left Behind Act, it was only a matter of time before test scores plummeted.
Guess what year that happened? :)
2015 it seems
I found this very interesting.
Do we have any research about NCLB? The Center for Educational Progress calls it the "worst education policy in decades" (https://www.educationprogress.org/p/schools-should-pursue-excellence). At the same time, that center has a specific agenda. I'd be curious to hear well-argued pro-NCLB points.
On the other hand, many commenters have pointed out that similar trends in test scores can be found globally. This would point at a more general underlying cause, not a US policy.
I just pulled up a graph of Ontario test scores in math and the trend from 2000 to 2018 looks very different from the USA.
I made the best argument for NCLB I could in my previous comment. Millions of students in failing schools got tutoring from private tutoring providers. This raised scores for two reasons. First, tutoring in reading and math is very effective. Second, school districts wanted to avoid paying for the tutoring so they focused on raising test scores for sub-groups of students who were normally ignored.
I am very grateful for your reply Jonas. Hopefully Scott Alexander will weigh in as well.
I am not an American, so all my information is second-hand, but I thought there was a consensus that NCLB made it so important for the least performing students to improve, that everything else (i.e. both the average and the best students) was sacrificed for this goal.
And then it turned out that the easiest way to slightly increase the results of the least performing students (because there is no known way to increase them dramatically) was "teaching to the test", which made things even worse for everyone else.
If this is true, then giving up NCLB should result in the least performing students getting slightly worse results, but everyone else getting much better results. So either this is not true, or it takes more time for the American schools to return their focus to the average and the best students.
Hi Viliam,
Thank you for your comment.
A few thoughts:
1) There are absolutely known ways to increase the achievement of low achieving students. Tutoring, for example, is very effective.
2) NCLB provided tutoring to all low income students in failing schools, including average and top students.
3) Teaching to the test without teaching the actual subject matter is pretty hard. Kids that can’t read don’t score well on reading tests.
I don't know about grade school, but at the university level (where I teach) there has definitely been some inertia in COVID-era policies that stuck around even after lockdown, analogous to the "stopped enforcing tough-on-absence policies" thing you mentioned.
The most prominent example I have seen is the removal of standardized testing: SAT/ACT for undergraduate admissions and GRE for graduate admissions. Lockdown is long past; in 2025 it is equally straightforward for a student to take these proctored exams in person as it was in 2019. Yet the tests are still not used.
Many such practices that started as "we need to do this, even if we don't want to, because with remote classes and remote work, we simply have no other choice", nevertheless continue now due simply to inertia.
But in some cases they continue because someone with an agenda used COVID as an excuse to enact the policy and now wants to maintain it post-COVID. They realize that it's easier to defend the status quo than to change things, even if the current "status quo" only just came into being during lockdown as an ostensible temporary measure. In other words they seized upon the COVID crisis to shift the Overton Window.
An example I could imagine in pre-university schools is this idea I had seen prior to COVID, but it was rare: the concept of setting a minimum of 50% for any grade. (see https://www.ascd.org/el/articles/the-unwinnable-battle-over-minimum-grades for example) I knew a high school teacher who worked at a school where this was the policy, and it was controversial among the teachers for obvious reasons. I have to imagine this policy is more widespread now due to COVID, and once in place, more difficult to roll back than it was to enact in the first place. (Just a guess though, I have no data on that.)
A few thoughts:
1. Teacher pay especially relative to cost of living, has declined since 2010, and the epidemic was likely just a wake up call to those on the fence that “essential worker” was essential the way slavery was essential to the antebellum south
2. Goodhart’s law predicts metric collapse, as the system tunes to a metric and eventually fails. It is possible that schools cut everything that didn’t seem to support the metric, then cut a few tendons too many
3. You have to factor in devices. Every teach I know says attention spans are nonexistent. You cannot read or do math if you can’t focus on symbolic decoding or think
4. The runaway 1% has increased anxiety among professionals and that has to include parents. Stress does not equal success
COVID can damage the brain. You don't think you're possibly seeing the result of letting a disease with systemic consequences rip and damaging children's brains?
"I predict that what we’re seeing here is not each individual child’s learning loss multiplied across all children, but a systemic effect where something about the pandemic made schools worse"
Or another possibility: schools' problems predate COVID and have continued. As you noted, the national trend started in 2017-2019. In 2014, the Obama Education Department issued a "Dear Colleague" letter that advised school superintendents that racial disparities in suspension rates would be grounds for finding school districts in violation of Title VI of the Civil Rights Act. News reports suggested that many schools responded by dramatically reducing efforts to enforce any discipline standards. To the extent this is true, it would make it harder for teachers to teach, at least in some districts with significant disciplinary problems.
Another item: it has recently been published that Connecticut schools have an explicit practice of never making any student repeat a grade for any reason - students don't have to learn anything to move on to the next grade, and eventually high school graduation. There was one notorious case last fall where a Hartford High School honors graduate sued the school because she can't read or write. I don't know to what extent this issue is reflected in other states, but it would support the idea that standards have declined, but not because of COVID.
I interviewed several teachers in 2021 and 2022, and nearly all of them told me that COVID had delayed not just their students' academic development but their social development and general maturity. One said "my 9th graders are really 7th graders". I've read others saying something similar. This would be an argument that the scores will recover, and that they should recover first for the youngest test-takers, then continue as they get older. However, the downgraded norms are being sticky; teachers who allow more misbehavior than they used to (they all told me they do this) may not realize they can raise expectations, and they'll keep getting students who have had other teachers who also didn't realize they can raise expectations, so they can't even raise expectations as much as they'd like (unless they have much more support for high expectations and consequences from parents, administrators, and the public)
This! Absolutely seconded from what I have heard from teachers in Germany as well. Only anecdotal, but they are very convinced that the students are less mature and less socially developed. I think all academic decline is downstream of that.
I am not so sure that this is *only* the lockdown. I think there was already a trend in that direction before Covid, at least in Germany. But the Covid period where kids are not socialized in a group (a long period, in Germany) has accelerated the trend a lot.
Yep, COVID leniency has been sticky. One of my local high school districts made 50 the minimum grade on all assignments and exams. You can literally do a third of the work and still pass.
Also during Covid BLM hit. School discipline declined because of equity, school to prison pipeline etc. Hard to learn when you cannot get rid of the badly behaved children.
There was no meaningful BLM in Europe but we see the same effect
I think the declines in majority white schools are essentially parallel to the declines in minority white schools
Social media and Chat GPT are the primary problem from I what see with children in Europe. COVID seems to have accelerated that. Children don’t learn how to read or write. Parents are also too distracted to get involved in their children’s education.
Interesting. I was wondering about this on international scale but hadn't looked it up.
Standardized tests in Germany suggest similar alarming trends of worse abilities. Experience surely does.
Not sure it's about COVID. Falling quality standards in schools is something I would also bet on from my personal experience as a parent of two kids.
That includes missed classes because of missing teachers (they are ill or their kids are)
And also focus on other things and minimizing (demonizing?) basic repetitive learning (I think it makes sense to some extent, but the result at this stage of education, ie 1st to 4th grade, is clearly bad)
Their school is considered quite good and my kids mostly like going there. But the basics are clearly lacking.
And I can't think of a parent who thinks to the contrary, also in other schools, so it's seems more like a systemic problem.
Not sure how much this is a mix of overparenting or self projection of our missed expectations onto our children... But we started taking those basics into our own hands for our first kid when we understood this and started directly with our second kid. And yes if course it's better.
Any possibility that COVID helped accelerate the saturation of internet and smartphone use, including normalizing putting a 3rd-grader in front of an internet-connected screen all day?
I don't know how localized this is, but where I live, there was a huge post-COVID wave of retirements and career-switches among teachers and other school staff, and since then there have been ongoing staffing shortages that show little sign of letting up. The staffing shortages lead to overwork and burnout among the remaining staff, and the rigidity of public sector wage scales means schools can't pay more to attract more workers.
Correlate it to the secondary, or recovery market price year over year for cellphones. I suspect another effect is that as old cellphones became cheaper the chance they were given to a younger child increased.
There was a very strong spike in recovery markets around 2019 just before Covid, and with Covid everyone had to get some sort of device to function. They may now be percolating to younger and younger children.
I think it’s either very young children are getting devices or specifically phones circulated around 2020/2021. Research has shown time and time again that before puberty children learn and retain the best from 1:1 face to face interaction with adults.
I gave my iPhones to my great great nieces and nephews as toys about 4 years after I upgraded.
Once you get addicted to your mobile phone it's difficult to quit
You've argued in the past that formal education as a whole is all but meaningless and now you're worried about a few points on test scores?
Not hugely worried, but I did feel bad that I wrote a post saying that people would recover quickly from COVID learning loss and the test scores seem to contradict that, and I thought it would have been cowardly to ignore this and not explain where I stood.
Also, I'm not sure I would describe formal education as "all but meaningless". I don't think it's very efficient or compassionate or well-done, and I don't think changes at the margin matter much, but the difference between going to school at all and not is probably important.
Ah, that's fair. Was just jarring given some posts I... vaguely recall (don't have the links and all) on unschooling and such, though thinking may have changed in the interim.
This is …well…😐
I feel like there are a lot of people in the Rat community who don't think school is that important, and I think it's partially because they generally didn't enjoy school very much and have intellectual elitism because they're more intelligent than their peers. (I'm also intelligent and got bullied at school.)
I can't be bothered to express myself properly, so I'll just say something crassly: being socialised is important, and I wouldn't a priori expect it to be pleasant.
> wouldn't a priori expect it to be pleasant
I would
Yeah? You go from an environment where you are coddled and have very little competition (especially if you are the eldest or an only child), and enter an environment where you are in constant competition and need to find your niche and fit in while maximising your status and future returns. That sounds hard to me.
Socialization can be done in various ways. I think the pleasant way is introducing friends gradually, starting with people who are (sub)culturally similar, and gradually expanding the variety. The unpleasant way is to be suddenly dropped in the middle of a mob that doesn't share your values. School does the latter.
Well, we usually socialise children starting at age 3-5 where things are a lot easier. Young kids don't think much about status games and are usually happy to play with any of their classmates. Kids get much nastier as they get older, peak in horribleness around 13 or 14, and then get better as they approach adulthood.
The worst possible case would be to be home-schooled until the age of 13 and then dropped into a school environment.
Unsurprisingly, this is essentially the plot of *Mean Girls*, though a little older, I think.
There's overwhelming evidence from PISA, TIMSS and smaller national testing programs that test scores have been dropping gradually since 2015 or so (maybe a little earlier depending on the dataset and country). Admittedly, the trend is not visible in US PISA scores - I don't know why this is, but it's common to most European and English speaking countries. This site has nice visualisations but only includes data for 2018 - the next iteration is being published at country level around now, and should eventually appear there. https://www2.compareyourcountry.org/pisa/country/oecd?lg=en
The trends are visible across multiple subjects, though perhaps strongest and most consistent in maths. It's common across more or less all countries (some South East Asian countries excepted). Covid might have accelerated the trend a little bit but there's a greater trend there which is crying out for explanation.
Some possibilities - I don't have evidence for these but they seem plausible: widespread adoption of mobile phones by children and adolescents; reduced parental involvement in educating children; declines in the quality of schools/teachers/curricula; environmental & health factors like microplastics, time outdoors, Covid. I *don't* think that there's a single neat explanation - maybe all of these feature, some more heavily than others in certain cases.
Been waiting for this...not mea culpa, exactly, but prediction-scoring after enough data came in. Sometimes it's not clear how to resolve these sorts of claims, but the ongoing barrage of omnipartisan "acktually learning loss was pretty bad" news has had me wincing about that 2021 article for awhile now.
Be curious what the test scores are like at the collegiate level. My company doesn't employ enough highschoolers to get much useful data, but there was definitely a bump during covid of flustered students who took a leave of absence from dysfunctional schools to get an Essential Worker(tm) jerb. (Not because they loved working in grocery, but because so many of the other common undergrad side gigs like waitressing got shut down.) So that'd be a, uh...I think that's a compositional effect? Where students who left and then came back didn't change grade levels, and are obviously academically rusty, so test worse by default.
My guess, once I learned that learning loss didn’t seem to be correlated with school closures, was that the stress and trauma of living through a pandemic had a negative effect on children that showed up on test scores. Can we rule that out? I do agree that the hypothesis that the schools got worse seems more likely.
Economic issues during Covid were a huge thing, also, both in terms of household finances and also educational cuts.
But if you're trying to control for American policy changes, wouldn't it make sense to use international data to establish some kind of baseline for comparison?
https://static.stacker.com/s3fs-public/mv9Tmpisaexamscoresfellbetween2018and2022DTMZ.png
https://radicalscholarship.com/wp-content/uploads/2023/05/uk-flat-pirls.png
Canada seems to share our downward trend pre-Covid. And ... what the heck is going on with Japan? Or non-Chinese Asian countries in general?
Perhaps many students were statistically less performative because they found themselves struggling with being either oppressed or privileged or born into the wrong body? Plus, there has recently been the distraction of democracy itself being at stake, along with the looming threat of WWlll, as well as there having been a surge in non-native, undocumented students entering the pubic education monopoly, with some of these students being girls having to compete in sports with boys while simultaneously having to overcome a culture of rape. Then again, perhaps the findings are due to the radicalization of parents becoming domestic terrorist threats. At least students’ free speech rights are still safely protected from hate speech and micro-aggression, despite low test scores.
Can we check demographics adjusted stats to make sure this isn't a Simpsons paradox
I hate to be so glib, but the inflection point for all these graphs seems to be 2017-2019. Could the DoE under DeVos cause this?
Originally I was flirting with "overall febrile disease burden, especially in the winter during these tests." We had those tripledemics after all... maybe it's not covid per se but the after effect of isolation and the vengeance of flu on immunity debt.
Or maybe just teacher enthusiasm, which surveys show has also dropped since 2019?
But the biggest problem with those theories is that when I look at the tests I'm seeing improvements on math and declines on reading.
Reading: https://www.nationsreportcard.gov/reports/reading/2024/g4_8/?grade=4
Math: https://www.nationsreportcard.gov/reports/mathematics/2024/g4_8/?grade=4
These are divergent outcomes! Anything about changes in culture or kids or teachers or kids explains too much.
So I went to the BLS time use survey looking for some indication that our culture has become STEM-pilled. There are declines in reading for men with big jumps for computer use.
2019: https://www.bls.gov/news.release/archives/atus_06252020.pdf
2023: https://www.bls.gov/news.release/pdf/atus.pdf
Still a bit of a stretch, the reading declines aren't there for kids in the time use survey, and "too many video games" doesn't match the divergent outcome so much.
I don't know what to think, especially with all the different assessments out there producing different trend lines. I guess if there are a bunch of assessments with different trend lines, maybe it's mostly noise and these will converge in another year or two.
Possibly related: "reading" as an activity becoming increasingly identified with primarily with 40s-ish women (source: the gift basket I won from my library for a reading contest over the winter, along with seeing what new books pop up on Overdrive/Libby)=reading becoming even less cool than it has been=kids wanting to read even less.
Of course, there's widespread reports discussing how now Zoomers are less and less literate and rely on video/audio for learning/and entertainment and are reverting to what looks more and more like an oral culture; if we assume that trend to continue (it's not like we're making less video/audio content these days, after all), one would expect reading to go down, which would also probably make people get worse at reading in general. Of course, the causality arrow might go the other way (people get worse at reading so they read less)
I think a lot of it is not COVID but BML/DEI that started in 2014, started peaking in 2017 and went into overdrive in 2020 after George Floyd protests. Schools in the best districts started to pay more attention to racial gaps and took... wrong actions. For example in our district https://www.bostonglobe.com/2024/12/02/opinion/newton-schools-multilevel-classrooms-faculty-council/
And what followed was the flight of well-to-do families to private schools that don't seem to be included in these metrics. COVID just accelerated it.
Not to say that covid-related closures did any good to kids either (I know a lot of smart kids that decided that it just doesn't matter)
Indeed. There were two huge cultural events in the first half of 2020. The Establishment has been trying to memoryhole the Racial Reckoning since, approximately, later 2022.
Good private schools are way woker than public ones. Just go on the websites of some famous private schools, they all go out of their way to show many black students they have and emphasize diversity. Sidwell Friends, where our political elite send their kids, the top article on their website now is about LGBTQIA+ and two others are about the Black Alumni Alliance and students staging a classroom takeover.
You are confusing good private schools with elite ones. There's a huge difference. Good ones might have bowed towards the DEI altar, because they felt like they had to but continued to have rigorous education "counseling out" kids who couldn't hack it.
Elite ones prostrated rather than bow.... but then, top-notch education was never their only (or maybe even main) goal. Their goal is to produce members of the elite.
This result seems to be global, not just American, so there must be some definite affect from closing schools and the way return to school was handled.
https://www.breakingnews.ie/ireland/junior-infants-not-coping-as-well-as-kids-born-before-covid-19-pandemic-teachers-say-1738505.html
"A new study from Mary Immaculate College (MIC) has found that current classes of junior infants may not be faring as well as children born before the pandemic, according to their teachers.
The research, ‘Infants of the Pandemic: Teacher Perspectives on the Early Development and School Readiness of Children Starting School in September 2024’, revealed that 81 per cent of teachers have more pupils now with emotional and behavioural issues compared to pre-pandemic times.
The online survey was completed by 107 junior infant class teachers from around Ireland (teaching over 2,000 pupils overall) during December 2024.
The survey aimed to find out if experiencing the pandemic during infancy might have influenced later development when starting school."
Junior infants is the age range 4-5 years of age; children are enrolled in primary school at age 4 in Ireland:
https://www.citizensinformation.ie/en/education/primary-and-post-primary-education/going-to-primary-school/starting-primary-school/
I think part of it is the same as the whole debate on here about "school is prison": for the bright, motivated kids and involved parents, lack of formal classes was less disruptive because those kids would continue to learn on their own/have parents making up the difference. Most kids would avoid doing homework, class work and the like if they didn't have to, and doing online classes was easy to skive off. Teachers had their own problems. By the time things returned to normal, the kids were accustomed to avoiding doing any more work than could be dragged out of them, the lost time wasn't caught up, and the accommodations (happened in Ireland as well) for less rigour around testing and grading just kept on sliding - after all, if you gave little Johnny a B for his work last year, why are you now reducing him to C or D when he is handing in the same level of work now?
You should probably talk to some school teachers, because they all seem to have a pretty good handle on the thing that's confusing you from your desk jockey chair. From the perspective of multiple high school teachers in Georgia, they identified three things with 2020:
1) the school lockdown nuked a year worth of learning progression, which is what you were talking about, but also
2) the BLM movement installed a lot of school board members and therefore administrators that were very much NOT about discipline in schools because (systemic racism) which led to the inmates running the asylum - higher rates of fights, disobedient kids, kids taking advantage of a system that would never punish them for bad behavior, and so forth,
3) cell phone addiction skyrocketed because parents used the cell phones as a babysitter during 2020 because they had no other choice.
So it was three awful things, not one, and two of the three haven't been unspooled yet. The process of unspooling them has begun, but cell phone bans only started to hit a few public schools in 2025, and they're difficult to implement, and BLM oriented school boards have to be voted out which takes a while, and in some areas will never happen.
Want to emphasize my agreement with (2)... The school I attend is in a fairly lefty area, but its administrators and teachers are even further to the left. (My understanding is that this is pretty common in K-12 education across the country.) After Covid, behavioral issues became far more common, and discipline became far less strict and far rarer.
Nowadays, if a middle schooler curses out a teacher or gets into a fight, they go through a "restorative" (fifteen minutes of saying they're sorry to the teacher or to the other kid while an administrator watches) and then... usually nothing else. Maybe a particularly bad repeat offender will be suspended for a day or two.
Scott wrote about academic and attendance standards slipping—in my experience, the drop-off in behavioral standards has been even more extreme and is probably closely related to lower scores.
I saw this a couple decades ago, when a counselor at our local elementary interposed mediation between a misbehaving kid, and the teacher who attempted to send him to the principal so that class could carry on.
I felt at the time this was an unfortunate equalization of the parties involved, and of course granted the kid power. What you describe sounds like postmodernism run amok; the kid will only feel that he’s been elevated, made important.
I wonder if this *ever* works as well as paddling once did.
Part of it, again, is that the worst parents are the ones who will scream the loudest about "you are picking on my little Johnny!" if the school attempts discipline.
They don't care if Johnny is learning anything, if he's a troublemaker; all they care about is that they aren't bothered by anyone, and certainly that they don't have to have Johnny at home during the day when they'll have to deal with him themselves.
"3) cell phone addiction skyrocketed because parents used the cell phones as a babysitter during 2020 because they had no other choice."
That's a minor thing arising in my country now, because the Department of Education is going to spend/spending money on special smartphone pouches so pupils won't be distracted in class (why the school can't just take the phones off the kids and put them in a cupboard until going-home time, I don't know):
https://www.rte.ie/news/politics/2024/1004/1473598-phone-pouch-schools/
https://www.thejournal.ie/department-of-education-seeks-tenders-for-e9-million-phone-pouches-in-secondary-schools-6598912-Jan2025/
There are lots of disparate things going on in the USA to address this as well. Some are seizing phones, some are throwing them in faraday cage bags, etc.
> why the school can't just take the phones off the kids and put them in a cupboard until going-home time, I don't know
When phones inevitably disappear, or are alleged to have disappeared, who will be responsible?
Though personally I think it would be simpler and cheaper to have a policy of, if we see or hear your phone, then it is permanently confiscated. Maybe you get a warning the first time. The school can sell the confiscated phones to raise money for school supplies or whatever.
Yeah, but if you're expecting the kids to put their phones in a locked bag and then carry that bag around with them, some of them are going to try and break the locks, or they'll lose the bags, or it'll be "my phone was stolen at school".
So? Mostly it'll work if the goal is to keep kids off their phones.
We need better methods of measuring reading proficiency. The low test scores could simply mean that students are worse at taking tests (less motivated, less focused etc.), not necessarily worse at reading or comprehension.
I was a middle school English teacher when Covid hit. In March of 2020, we suddenly went remote and stayed that way for the next 1.3 school years. The way I think of it is that Covid remote teaching broke kids brains because they peeked behind the curtain and can't unsee what they saw. Imagine being in a relatively orderly, functioning classroom one day, with a whiteboard and desks and other students around you, and the next day (and seemingly forever) you're then in your bedroom and you can turn your camera off and you can play games on the computer and you can pretend your audio is not working and you can't hear.
The dynamics completely changed. Kids realized, school as I knew it is not a fact of life. Learning was mediated by one small flat Chromebook screen that made school 100X more boring than it already was. Sleep habits, already book, took a nosedive. You're doing everything in one space. Your bedroom is your classroom so the vibes of sleep and learning blend together. You get used to doing things your own way, being able to mute everyone, not having to participate... That is not good for mental health. Plus, young kids on computers are getting poor digital habits early. Also not good.
It was a shitstorm and it's still unfolding. Just go to r/teachers or r/teachersintransition for a little look-see. I left teaching at the end of 2021, mid-year, to join my wife in her business. This was one semester into the first school year back in the classroom. It was complete insanity. Kids constantly talking, causing chaos in the bathrooms that no one could ever catch, doing next to no work, mental health of everyone bad and getting worse. I understand these trends have only continued.
School was already antiquated and so bad in so many ways. This just pulled the mask off so everybody could see — especially the kids. I love kids. I feel terrible for them. We have bureaucratic lock-in of a broken system that breaks more and more young brains, all the more so since Covid. Ugh. Yeah. Dystopian.
A lot of that is pushing back responsibility onto parents - if they were not/could not monitor their kids at home to make sure they were attending the virtual classes and keeping up with the work, of course the kids learned bad habits.
The school system as presently constituted may be broken, but the lockdown showed that without *something* in place, education will fall off a cliff as regards the majority of school-aged children.
One aspect not really brought up here or in your original post is that schools have a very different role for lower class children, where they serve as a key distribution point for most social services. Just as one data point, about 12 million children relied on schools for breakfast a decade ago. For this population, schools also fulfill more abstract needs, such as stability or functional adults. So school shutdowns should also be considered equivalent to shutting down most government support targeting children or families. It's possible that this had more of an effect than eliminating the actual pedagogy.
The only allusion to this issue in the original post I found was "if you come from a really bad household school is the only place you can learn expected values and behaviors". This issue and removing basic services such as food could account for a lot of the drop in scores. You do point out that there's conflicting data on low-performers specifically, but that's to be expected. Schools serving as social services hubs with pedagogy as a superficial side-benefit is inconvenient for most political narratives so is rarely even considered and the definitions and assumptions used in research aren't necessarily well-suited to show it.
How much is a 2-point difference in scores (from 216 down to 214)?
The difference looks significant in the state-level bar charts that Scott included. At the same time, it's only about 1% in relative terms.
Wikipedia says that "Main NAEP assessments are typically administered over approximately six weeks between the end of January and the beginning of March of every year." Part of me wonders if February 2024 happened to have particularly strange weather (It was the warmest on record). Or whether there is another unknown factor at play here.
Just look at that positive outlier. What did Louisiana do *right*?
1. Training teachers in research-backed methods for teaching reading.
2. Holding back third-grade students who are significantly behind level, thereby excluding them from fourth-grade reading-assessments.
Does (1) refer to phonics? I heard there was a reform in that direction.
Good for Louisiana.
I think it’s more than that, but that’s a big part of it.
That second one is colossal. Kids aren't being held accountable, they're not being given fair grades or being held back. So they don't care.
I have another hypothesis to propose: current economic conditions make it impossible for parents to monitor their child's educational progress as closely as before. Covid, where shut downs put more families under greater economic strain, made this trend worse, but it started before then.
If correct, this implies that the school itself is less important than family circumstances. Certainly, families in impoverished areas are both exposed to schools that didn't have the resources to cope with the effects of Covid, but that isn't a perfect relationship. And it wouldn't just be poor families experiencing this pressure, but ones employed by businesses more greatly affected by the pandemic. I would like to see a chart for students who are the children of healthcare workers.
I remember my parents knew when I skipped a single homework assignment. They wouldn't let me out of the study area until I had completed each one, every day. I do the same with my children, but I wonder how common this is anymore.
Seems more likely that parents had more say over their kids' schooling during the Zoom Era.
Why? If by "Zoom era" you mean the shutdowns due to the pandemic, well, I was a parent of school age children then, and that simply isn't true. I was on my own zoom call, doing my job.
Lots of people said they noticed more about their kids' education when they were all at home together.
I can only attribute that to parents who previously had known nothing. And I can only attribute that to not caring. If you are the kind of parent who takes the time to become invested in your kids education, then in person classes aren't the barrier. The real barrier is the time and energy the parent has.
> Re-reading the post, I still think my arguments make sense.
This chart makes it seem like scores have been continuing on a pretty consistent trend since before 2019, with no change at all visible during covid. It's actually kind of surprising how little evidence there is of covid having any impact at all. Why would you assume there's anything going on here other than the continuation the pre-existing trend? This "trend" is based on 2 data points, true, but it follows a long plateau, so it could make sense if there's some longer-term trend slowly worsening education, and it's still the case that's there's really nothing to *explain*.
Our ability to build complexity has gotten too far ahead of our ability to live with complexity on net. We are slowly making our lives worse. Overall human performance is starting to degrade. Covid just forced a breaking point that was coming soon anyway.
My wife is a primary school teacher in Switzerland’s largest city, where there are many immigrant students. We have similar discussions here about the decline in reading and math skills.
Reading:
A key issue is that the language of instruction in schools is not the native language of many students, which naturally affects their reading proficiency. But beyond that, deep reading itself seems to be becoming an outdated skill. For most people, there’s little pressure to reach a high level of reading ability anymore—neither for daily life, nor for entertainment, nor even for many entry-level jobs.
As someone who has avidly read both fiction and non-fiction for decades, I’ve noticed a shift in my own habits. I now consume many books via text-to-speech on my e-reader app and get much of my information from podcasts and YouTube—sometimes actively watching, sometimes just listening in the background. Recently, I uploaded a non-fiction book to NotebookLM and listened to an AI-generated podcast discussion of it instead of reading it, because I found the author's repetitive writing frustrating.
I barely use Google to find and read through websites anymore—I just ask ChatGPT. Kids today do the same, relying on AI tools, TikTok, and podcasts. It seems that reaching a truly advanced reading level is becoming a niche skill, rather than a widespread necessity.
Math:
I’m less certain about the causes of the decline in math skills. One possibility: In the past, highly competitive Asian "tiger moms" might have pushed their kids toward U.S. degrees, but with the West losing its economic and technological edge, perhaps more of them are choosing other countries or staying in China instead. But I don’t know—just speculation.
PS: as a non native english speaker I drafted a raw version and let chatgpt do the rest...
I have definitely noted both a decline in written English (a lot of writing that spells words 'as they sound' and it's clear the person hasn't seen it written down, or at least their schooling never corrected it; one thing that seems to be becoming commonplace is using "free reign" instead of "free rein") and increasing use of AI to generate online media, even for 'traditional' outlets (e.g. a news article that used "site" instead of "cite").
I don't know what the solution is, but if we have increasing use of AI that is poorly educated to generate output to look forward to, the future will get into a vicious circle of 'incorrect/bad usage, people see this and think it's right, they use it themselves, the AI gets trained on the corpus of output from these people, it uses the incorrect/bad usage, people see this...'
Oh well, time to go and yell at those kids to get off my lawn!
Fate of empires depends on the education of youth
Aristotle
This is obviously tied to a fear of enforcing rules, especially on poor (i.e. brown/black) students post-2020. Similar to the general decline in policing (in a general sense) and strong indicator (imho) of the usefulness of a more paternalistic approach.
If that is true, wouldn't you expect to see divergent effects by race? But the effect seems to be there regardless of race, or school demographics for that matter.
I would like to see if there is a correlation with increased smartphone/tiktoc use among kids…
Could this be related? You would expect it to worsen test scores and the timing roughly matches up
https://www.pbs.org/newshour/nation/obama-administration-recommends-ending-zero-tolerance-policies-in-schools
> Holder said the problem often stems from well intentioned “zero-tolerance” policies that too often inject the criminal justice system into the resolution of problems. Zero-tolerance policies, a tool that became popular in the 1990s, often spell out uniform and swift punishment for offenses such as truancy, smoking or carrying a weapon. Violators can lose classroom time or become saddled with a criminal record.
I gotta say -- my overall impression of zero tolerance policies has not historically been favorable... but also, is it not obvious that the things in this list are not the same??
Could we perhaps cut students a little bit of slack for skipping school, and try out some restorative justice or whatever, but still get the police involved if they are carrying weapons? This seems pretty obvious to me. And there are certainly a lot of anecdotal reports of "zero-tolerance" policies being applied to things like "fighting", in a way that gives severe consequences to both attackers and victims. So many of these implementation details just seem insane to me. I don't know what that says for the concept as a whole.
That's part of it, sure. There's an entire mosaic of different reasons why schools succeed or fail, and one element will be "disruptive pupils can't be disciplined effectively". If you have a bunch of kids who will be expelled from one school and bounce around from school to school, then there will be one final school that gets lumped with them (because education is a right and parents can sue if their kid isn't in school).
Since the kids have absolutely no interest in learning and probably have a raft of behavioural problems as well (future criminals in the making), plus the parents only care about "get them out from under our feet during the day" and not a genuine education, these schools are just holding pens until the kids are old enough to leave.
Well-meaning intentions from the top-down then imposed on the holding pen schools about "you can't expel them, you can't fail them, that's racist" means that they'll just be shuffled through the system, permitted to graduate, and dumped out on the streets because now they're somebody else's problem.
And the academic grifting movement about systemic racism declaring that "more black kids get punished in school than white kids, this is racism, so dismantle all disciplinary structure and expectations of academic achievement - because maths is racist too" just perpetrates the failure.
It's not the entire reason, but it's one more little tessera in the whole mosaic.
Put your predictor hats on! What do *you* think the 2025 or 2026 NAEP scores will be?
(If you want to look at the aggregate data it's at https://www.nationsreportcard.gov/data_tools.aspx . Do note that if you want *individual-level data* you will need to ask for a "restricted-use data license" though.)
Nonexistent, because Musk will have fired all the people who work on it.
I want to know what the hell happened to math in 2012/2013? Scores for most groups started plummeting after that. Was there a change in the tests? A change in the way math is taught?
Unfortunately, this analysis neglects the impact of Skibidi Toilet.
I agree with the comments that the decline started in 2015, not with Covid.
Potential contributing factors;
1. Steady long term increase in non-white and non-Asian students in NAEP data. Other groups have significantly lower scores (per NAEP) and this is establishing a “headwind” for scores.
2. Movement of better students (often also white or Asian) out of public schools for obvious reasons. Another headwind. (I live in California and the number of responsible middle class parents that I know who leave their kids in public schools is really tiny)
3. Increase in single parent households.
4. Less reading, especially after 4th grade. More screen time and distractions.
5. Bad habits acquired during Covid by students, teachers, parents and schools.
6. Lowering standards in schools related to new teaching methods, diversity efforts and movement to pull down honor students for equity?
7. Movement away from phonics?
Even if all these are a little true, and some are indisputably true, the question still remains, not why scores started dropping after 2015, but more importantl, why did they go up for the 15 years prior? What changed in 2015 to no longer allow us to overcome these longer term headwinds? Does it have something to do with changes to Leave No Child Behind?
Said more concisely, we have had demographic headwinds for the last few generations, yet scores had been rising up until 2016. What changed?
You can look up NAEP scores by demographic group:
https://nces.ed.gov/nationsreportcard/
All good points.
My personal bias is to look at smartphones as an explaining factor. Smartphone unit sales peaked in 2015. The second derivative (rate of change in sales growth) turned negative in 2012 i.e. reaching saturation point. I would argue that any large social changes that occurred at/around that time interval should be checked for links to smartphone usage.
Covid lockdowns didn't just temporarily disrupt kids' schooling. They also sent a message to kids that school is just not that important. That message was received loud and clear, and has done lasting damage to learning culture. This essay is a great example of that: https://kappanonline.org/gaither-russo-how-the-pandemic-response-destroyed-the-learning-culture-in-one-baltimore-high-school/
There is bound to be many reasons for this. One thing to consider is that zoom school wasn't a good fit for many students. For many kids the interaction with other students and the dynamic flow of events in the classroom are as important to their learning as the teacher's information dump. Zoom school was flat, like a can of soda left open overnight. It lacked the flow and interplay of classroom learning and, as a result, many kids lost their sense of connection and engagement with school. The world as they knew it ended, and school as they knew it lost it's luster. A lot of kids lost their sense of academic purpose and direction and it's not easy to reinvent that.
Or it might possibly be some of the best evidence yet in support of Sheldrake's theory of morphic resonance.
I think people not involved in education don't have a real idea of the work teachers do. It's not just information dumping, sometimes it is (metaphorically) spoon-feeding the pupils of lesser ability so that the teacher really is dragging them all the way over the line.
If you don't have that real-world interaction, a Zoom classroom is not going to work.
I used to think this in connection with the art teacher at my child's school. It was so physically demanding a job. She had them do stuff in several media, and in various "genres", year-round. The kids made the most beautiful and/or eye-catching products.* And they really did work on them, of course - but also it was plain to me that the teacher made sure that every last one was completed and decent-looking. There was much less of an effect where you could notice who the "best artists" were and who the worst.
I used to substitute teach and I rather dreaded "art" class because it was so much work!
I remember being in one such class all day and the teacher had deliberately left the easy assignment that the kids were merely to do some free sketching or brainstorming to ready their subjects for whatever the latest project was. A little girl came up to me and asked me to draw some animal for her. I had to confess I was no more proficient at drawing that animal than she would be.
"But you're an artist!" she said.
"No, I'm just a sub" I said.
"You have to be some kind of artist" she insisted, incredulous.
*It makes me laugh to think of my early grades spent in a Baptist church school. Their idea of art was e.g. to give us first graders boxes of toothpicks and glue. There you go. Make something. For a month when it was art time, we would drearily get our "tray" out and re-commence our toothpick work. A pile of toothpicks glued together. Then at Christmas we were asked to bring a phone book from home. We lived in a big city with an enormous phone book. Our art task was to fold each page of the book down. You read that right. After a week or two of this folding, our teacher opened the book out and spray painted it green as I recall - a Christmas tree. I don't remember any further decorating, perhaps we had folded too slowly and it was now past Christmas.
In your grade 4 reading graph, Oregon had the second biggest fall in NAEP scores, and I think that tended to be true across the different flavors of NAEP tests (reading vs. math, 4th vs. 8th grade). Oregon's public schools got extremely woke, perhaps even before 2020, so Oregon's performance appears to be, basically, a self-inflicted wound.
In general, it's worth remembering that the the pandemic coincided with the "racial reckoning" that began on May 25, 2020, which led to higher rates of homicides and traffic fatalities due to more lackadaisical policing. So, it's hard to disentangle the effects of covid and of the George Floyd racial reckoning on things like school grading standards.
In general, students from the upper half of society, those who don't get subsidized lunches, didn't seem to be hit all that hard by the events of 2020, judging from NAEP scores, just as their parents tended to perform their laptop jobs not too terribly during Work From Home.
But students from the the bottom half of society (those with free or cheaper lunches) got hammered on the NAEP, probably by being expected to study on Zoom. It's almost as if those kids tend to benefit from going daily to an organized place called a school where middle class grown-ups talk at them all day.
This is broadly the argument I make when the usual "school sucks and I hated it and it's prison for smart kids and I wish I had not been forced by the educational Gestapo to attend" conversation happens on here.
If you're smart, and if you have involved parents, then school probably doesn't make that big a difference to you.
And no, "now we can all do learning online so we don't need teachers and physical schools" is not a solution. Same applies: if you're smart and have involved parents who guide your learning so you're not slacking off or neglecting the subjects you don't care about, then online learning is fine.
If you're smart but poor/parents aren't involved (for a whole host of reasons, ranging from 'badly educated themselves so can't help even if they wanted' to 'you would in fact do better were you raised by wolves in the forest'), you may or may not be able to make up the difference.
If you're not Big Brain I Trounced The IQ Test kid, you probably need school discipline and structure. Throw in the worst of all worlds (poor kid, not academically in the highest range, uninvolved parents) and you are likely also getting free meals, guidance counselling and the host of support services that schools are now expected to provide - in effect, being quasi-parents not just educators.
There's only so much schools can do, when the kids go home and nobody gives a damn if they learned to read or not, and education is seen as some dumb imposition by society because it's never going to matter to you or anyone in your family.
And what Covid and the lockdown showed us is how the cracks really are there, and how kids can fall through them, and that no, not all kids are self-motivated little firecrackers who will be doing deep-diving on academic subjects if they are just allowed freedom and no mean old school forcing them to sit at a desk.
I think - as one who found school so intolerable, for reasons both of boredom but also my problematic temperament - I can without hypocrisy say that I believe we all would have flourished more in an environment that was closer to summer camp than to being uncomfortably ensconced for 12 years under flourescent lights in what was essentially indistinguishable from Class C office space, only worse.
Even a beautiful school would have made a difference to me.
I skipped school in late high school (decades ago) not because I was a bad student but because I was a good student. My school (in Toronto) was set up in such a way (terms instead of semesters, final exams only for mediocre and bad students, well-known grade fixing for better students in the humanities in OAC) that there was no reason to go to class every day after a certain point each spring. And the consequences were minimal, especially once I turned 18. Obviously mine was a very unique situation but there could be weird explanations for absenteeism that aren't "not good at school"/"don't like school."
But, on hand/eye coordination, we're through the roof!
From its begining NAEP has been a kind of Rorschach. Like many social indicators that's inherent in its design. You can offer more or less plausible hypotheses to explain the differences you see cross sectionally in a particular year or over time, based on associations with the "background" information NAEP gathers and reports on the students and schools sampled. And as the comments here demonatrate you are free to add whatever speculations you wish based on personal experience, ancecdote, and policy preference or bias. But you really can't make valid causal inferences based on NAEP data alone. The designers work very hard to try to make as many "construct irrelevant" variables, such as the test administration conditions, etc. etc. as "equal" as possible, but you never know for sure. Their official materials do caution you about the limitations, but I think the various "theys" are quite happy if they can provoke enough speculation to keep us paying attention.
Early NAEP didn't report in terms of performance scales - they just provided percent correct for specific items or clusters of items by various demographic categories. That was in part because of Tyler and Tukey (the original designers - an educator and an eminent statistician) -Tyler didn't believe in scale scores, and Tukey thought he could sqeeze all of the information he needed out of the item performance data. In those early days it was really very hard for anyone to make "What's it all mean?" sense out of the raw results, and it was in real danger of dying from loss of interest. NAEP didn't report in terms of psychometric scaling until an evaluation (one which I had some responsibility for, and now feel a bit guilty about) led to its being moved from the Education Commission of the States to ETS. ETS did its fancy thing and began reporting in terms of distributions of performance by students in various demographic and location categories on numerical scales with a standard range (like SATs) for the three age/grade levels of schooling - based on "matrix sampling" of students and content clusters. It is important to recognize that no student takes a full "NAEP Test" in any subject. The distributions of individual scores are distributions of "stat-rats," not real kids. They make a pretty rigorous case for why that particular fiction is o.k. for their purposes, but you really need to recognize that you are not looking at something like individual scores on an SAT, or for that matter on a State required accountability test, in most states -- so causal interpretation of the "individual" scores does get pretty "quirky" - which is a partial explanation for what Scott noticed as he looked deeper (?). Over its history NAEP found that it was still hard for the public to know whether they should care about the scale scores and changes in them over time, so NAGB (the governing board) proposed, and Congress allowed, on a "trial basis," reporting of scores in terms of "Achievement Levels" - i.e. "Below Basic, Basic,
Proficient, and Advanced" based on a rather loose eyeballing of the items that seemed to be associated with particular scale score levels by selected groups of Subject Matter "Experts," Educators, and Citizens (??) - so that NAEP basically could tell the public what was good, better, and not so good, and lo and behold, NAEP became "the Nation's Report Card." Those levels have never been sufficiently "validated" so Congress could be convinced to drop the "trial" monniker, and critics have often suspected that the levels are arbitrarily set high enough so that the public might always be encourgaged to view the results with alarm, and thus justify either sending more money from the Feds to public education or maybe abandon public education entirely - pick your
preferred conspiracy ..
My real point is -- you don't really know what these things measure or how much you should care about a 1 to 10 or 20 point change in any direction. They are consistent enough that they probably are measuring something, but how closely that something is related to something you care about and that you, or the schools, know what to do about, really requires a much closer look at actual curriculum and actual instruction in practice. And if you can do that, maybe you really don't need these slippery numbers at all - just get on with teaching and learning. Enough with the remote
control!
Results from the Central Statistics Office (in Ireland) about their survey of the public on the effects of Covid - this is the education part.
Parents and students alike felt it had a negative impact, and those from poorer families rated the impact more negatively.
All the data here:
https://www.cso.ie/en/releasesandpublications/fp/fp-c19ie/covid-19-ourlivesfiveyearsoninterruptededucation/
dysgenic policies from the 2000s are now showing up in the data.
Any consideration of whether COVID just directly caused sufficient brain damage to influence outcomes?
High kids stay high, low kids stay low, this is the immutable and truly important point - students don't change places in the distribution relative to cohort
In addition to the ways that liberals are screwing over teaching a trend from conservatives is to reject any local tax increases, no matter how bad that is for the schools.
The plateau (and falloff) around 2007-2011 is of much more interest as far as I am concerned. I'm not sure what happened there because American public schools have been a steaming hot dumpster fire for a *much* longer time than that.
In many states, the standardized state assessment tests have been regularly (and purposefully) changed so that the decline isn't as noticeable. Obviously educrats understand the meaning of standardized in a very narrow, changeable way that allows them to massage data and claim improvement by changing the test whenever things start to look awkward.
For what it's worth, Steve Sailer published his take on the new NAEP scores today. However, a good deal of it is behind a paywall.
https://www.stevesailer.net/p/naep-test-scores-mississippi-miracle
You can get behind the paywall if you sign up for a subscription. That gives you a seven day grace period to poke around the archives. But if you don't cancel before the free trial runs out, you will be charged.
I am waiting with baited breath for Freddie DeBoer's response to this.
My guess is a few structural factors, a mild covid effect, and massive cheating all around.
It could also be increased absenteeism because children are being infected repeatedly as schools refuse to integrate covid precautions such as cleaner air, and forcing students to remove masks, harming their immune (and other) systems, causing long-term sickness. Teachers are also increasingly off sick, which no doubt has an affect on the children's learning.
Is Louisiana cheating on the test?
I see this very differently, and believe Covid is a key component. If you follow the copious medical research on the long-term post-viral impacts of Covid, along with rising death rates and sickness of people, you see that it’s not a ‘culture of absententeeism’ that is to blame but that people are actually sick a lot more. Add Covid health damage (including cognitive damage) to this prison-like environment and high stakes testing requirement of our schools, plus overloading teachers, and we have an education system that is collapsing.