Paper That Claimed Closed Schools Cost Lives Gets Correction – NPR

In November, I reported for NPR on a scientific paper that estimated millions of years of life could be lost due to prolonged school closures in the U.S. — far more, in fact, than might be lost by keeping schools open. The paper has since been corrected and critiqued. The central question it tried to answer remains.

The paper’s author, Dimitri Christakis is a pediatrician at Seattle Children’s Hospital, editor of the American Medical Association journal JAMA Pediatrics and an outspoken advocate of opening schools when possible to protect children’s well-being. He told NPR recently that he wrote the paper to flesh out his argument:

“The debate has been around kids going to school or life lost. When it’s framed that way it’s a no-brainer. But we’re also killing people by not putting kids in school. It’s just that they’re not dying today, so we’re not taking them into account.”

Killing people? Because of several months of Zoom school? How could that be?

Well, there is a well-documented association between educational attainment and life expectancy. By young adulthood, according to one study, Americans with a college degree can look forward to a decade more of life compared to people who don’t have a high school diploma. This gap tends to be larger in more unequal societies, like the United States. The Organization for Economic Cooperation and Development reports the average gap across rich countries is six years.

But could temporary school closures really hurt a significant number of young Americans’ chances of graduating college? It’s also been documented that even short interruptions in education — especially when they’re combined with social and political upheaval, such as we’re currently experiencing — can knock some students off track to their degrees. It can take years to recover from just a few months of lost learning, and some never do. Already, NPR has reported college enrollment plummeted this past fall. That’s a reversal of what has happened in previous recessions.

Taking these well-established patterns into account, Christakis’ paper calculates that we are very likely losing more years of life by keeping schools closed, than we would have lost to COVID-19 if we had kept schools open:

“This estimated loss in life expectancy was likely to be greater than would have been observed if leaving primary schools open had led to an expansion of the first wave of the pandemic.”

The paper received a fair amount of attention. It soon caught the eye of

Gideon Meyerowitz-Katz, an epidemiologist at the University of Wollongong in Australia, and Ilya Kashnitsky, a demographer at the University of Southern Denmark, who Christakis calls “two trolls, frankly.”

Trolls, because they concluded the paper was wrong. Really, really wrong.

“The most surprising thing about this paper is that basically every assumption … that they apply in the paper contains errors of some kind,” Kashnitsky told NPR.

Meyerowitz-Katz and Kashnitsky published a preprint (not a peer-reviewed paper) to that effect, with the spicy title: “Comparing bad apples to orange soda: Flaws and Errors in an Estimation of Years of Life Lost Associated With School Closures and COVID-19 deaths by Christakis, Van Cleve, and Zimmerman.”

The details are a bit in the weeds. Among the errors that Christakis corrected and responded to, out of many detailed by Kashnitsky and Meyerowitz-Katz, there was a transcription error and a calculation error.

But that’s not enough for the two, who maintain that all the errors taken together reverse the main finding of the paper and that the whole paper should be retracted.

“If you check the corrected version now, it’s even more ridiculous than the initial paper,” Kashnitsky says. He is not the only one who feels this way. A recent Zoom meeting of a scientific group called “The Decision Sciences For Child Health Collaborative” included a long discussion of criticism of the paper, its assumptions and its methodology.

Christakis points out that the journal stands by his paper. Again, he is the editor of JAMA Pediatrics; this paper was published in another JAMA journal. “The paper is based on two unassailable truths,” he says. “One is that education improves health, and lack of education negatively impacts health, and that distance learning doesn’t work very well, if at all, without parental involvement for primary school kids.”

The other truth, he says, is that more and more studies have shown that schools can be open without contributing greatly to the spread of the virus.

On this last point, Kashnitsky grants, Christakis may be right. He’s a demographer, so this isn’t precisely his area, but he notes that in Christakis’ paper, in order to estimate how many deaths were avoided by closing schools, he averages the results of two studies. One of them found that school closures by themselves had almost no effect on the spread of COVID-19. The other study suggested the opposite, that school closures, all by themselves, prevented 40,600 deaths from the coronavirus. Kashnitsky says the studies’ conclusions are so far apart that one of the two almost has to be wrong — and his judgment, at least, is that the first one “looks more solid.”