Andrew Gelman picks apart the study behind a front page New York Times story. The story also got a link (and an uncritical link at that) on Freakonomics.
Gelman seems flabbergasted at what he sees as a hyperbolic claim resulting from a poor design filtered through poor journalism. I’m surprised at the surprise. As one commentator “Mark” points out at Gelman’s blog,
Andrew, this is my area of research (public health), and I don’t think you’re missing anything, and I’m not the least bit surprised that it resulted in big headlines in the NYT. This happens ALL THE TIME. Recall the recent results from Harvard regarding red meat and cancer: http://www.nytimes.com/2012/03/13/health/research/red-meat-linked-to-cancer-and-heart-disease.html?_r=1. OK, this one wasn’t exactly front page in NYT, but still they were hyping another similarly hopelessly flawed study. And, it’s not too hard to find many more examples in the NYT, some of which certainly made the front page. See John Iaonnidis.
Another commentator “Jonathan” suggests, “Anyone know of any studies that look at impacts of new studies on behavior changes. It would also be interesting the difference before and after the introduction of the internet. If that makes sense.” I second that–can we get data on gym memberships and match it to NYT circulation??
The headline here suggests yes. But the article gives no evidence on this, and instead spends most of the time giving evidence about the possible link between vaccines and side effects. Here’s the only suggestion of elite leadership I found:
Historically, Dr. Willoughby said, vaccine scares have caused vaccination rates to drop for three or four years, and have led to outbreaks of diseases that had previously been under control, like measles and whooping cough. Measles cases in the United States reached a 15-year high last spring, with more than 100 cases, most in people who had never been vaccinated.
Even if the Times had a plot showing a correlation between elite statements and vaccination rates, causality wouldn’t be shown. The basic problem is that elite rhetoric could anticipate, instead of echo, what the mass public thinks. This issue comes up a lot in public opinion research. Maybe vaccinations and these types of statements are a good way of testing this.
Weaving nutrition and exercise lessons into middle-school classrooms can reduce eating disorders among girls and ultimately save medical costs, a study by Boston researchers concludes.
The researchers analyzed data from an earlier study at 10 Massachusetts middle schools, including five that adopted an obesity prevention program called Planet Health, and five that did not.
Austin’s team estimated that if the obesity-prevention program they studied was expanded to 100 schools, there would be about $680,000 in health care savings.
From the article “Nutrition lessons for girls seem to pay off” in the August 2 Boston Globe. Commenters note small sample size and that this seems like an ad for “Planet Health” instead of an unbiased article.
RESEARCHERS AT the University of California, San Diego, use questionable data to make the flimsy argument that a few sips of alcohol are enough to significantly increase the likelihood of serious injury in an automobile accident (“Study: 1 drink raises driving risks,’’ g section, June 27).
Their data only reflect numbers for accidents in which there is a fatality. That’s only three-tenths of 1 percent of all car accidents – 34,172 out of 10.2 million. It’s like trying to discover how widespread steroid usage is and using Major League Baseball players as your sample.
I’m too tired to evaluate this. Any takers?