Monthly Archives: April 2011

Does happiness lead to suicide?

The Associated Press (via the Washington Post) reports the “results” of an experiment linking macro-level happiness to micro-level suicide rates. “The surprising result: The happiest places sometimes also have the highest suicide rates.”

And yet, the studies author stops short:

But Wu urged caution in drawing conclusions, saying: “I don’t think that means if you are unhappy you should be around others who are unhappy.”

My question is, why? The article doesn’t say what problems there could be with this study. I could think of some, but just want to point out that the journalist strangely doesn’t pursue the researcher’s caution. Foremost, the article just tries to demonstrate a correlation, apparently thinking that is a good first step toward showing causation. However, it is known that correlation is neither necessary nor sufficient to have causation. See, for example, here.

Are technology and multi-tasking driving us to distraction?

1 Reply

The Boston Globe would have us think so. And so would, apparently, some professors at my school, MIT:

While Inman took matters into her own hands, some MIT professors are urging college leaders across the country to free students from their tether to technology. Over the past decade, schools raced to connect students to the Internet — in dorms, classrooms, even under the old oak tree. But now, what once would have been considered heresy is an active point of discussion: pulling the virtual plug to encourage students to pay more attention in class and become more adept at real-life social networking.

“I have been a bit skeptical about the value of making an entire campus wireless,’’ said Lawrence Bacow, president of Tufts University and former chancellor of MIT, where he was a professor when it began wiring all classrooms in the mid-1990s. “It seems like everyone is always plugged in and always distracted.’’

Ok, so I’m reading through, looking for some data. Anecdote… anecdote… anecdote… aha!

Indeed, a 2009 Stanford University study showed that students who were chronic media multitaskers were more easily distracted. Not only did it impede students’ concentration and learning, the effects linger, said Clifford Nass, a communication professor who embarked on the study after noticing freshmen would write papers in their dorm while simultaneously chatting with multiple friends on Facebook while also talking to someone in their room and surfing the Web.

Selection bias, anyone? No worries, we can just fall back on the time-tested change-the-outcome and change-the-subject approach:

Even more worrisome than the negative effect on academics is the social consequence, said Nass, whose current research focuses on the emotional implications. Preliminary evidence shows that college students who multitask are less emotionally attuned to others, he said.

On a similar study, see here. I won’t believe any of this research until I see multi-tasking randomly assigned. I’ve seen one experiment, but think the method was flawed–see here.

What matters more for economic outcomes: hard work or class?

Leave a reply

Another way to phrase this question may be more common: do we live in a meritocracy? Arthur Brooks, head of the American Enterprise Institute, certainly thinks so. Here’s an excerpt from his piece in the Washington Post a few days ago.

And so it is in our country. If opportunity in America is a sham — if the system is rigged and some people get the breaks only for reasons of luck, birth, or discrimination — then merit is fictitious and redistribution brings greater fairness. But if America is an opportunity society — if you have the chance to work harder, get more education and innovate — then rewarding merit is fair, and it is fair for some to make more money than others.

Brooks pulls selectively from the economics research to prove his point. But in the end, I’m not sure he’s even convinced himself. For example, he seems–and this is becoming a general theme of my critiques on this blog–to waiver between rejecting we can know anything for certain and forcefully arguing that his own view of the causal relationship is right:

Since equality of opportunity is not universal, doesn’t this invalidate — or at least weaken — the romantic notion of meritocratic fairness? Of course not. You’re living in a dream world (or you have tenure) if you really believe merit doesn’t matter. Everyone can think of times when things went well as a direct result of hard work. We can also come up with cases in which we were punished at work or in life for laziness, incompetence, free-riding or stupidity.

Yes, but what’s the systematic relationship between these things? Brooks hedges, and instead blows some ivory-colored smoke:

Most important, if we reject the ideals of opportunity and meritocratic fairness, we will end up with a system where outcomes are simply based on luck or political power — it would become a self-fulfilling prophecy. In a 2005 study published in the American Economic Review, economists at Harvard University and the Massachusetts Institute of Technology studied 29 countries and showed that a belief in luck over merit was strongly linked to the level of taxation and spending on social programs. Furthermore, they showed that the more citizens believed in a merit-based system, the more their public policies produced such a system.

I don’t care where these folks were from, but you need to offer more evidence than a correlation cooked up by some anonymous people from big name schools. And by the way, what was our causal question again? Note that now we’re thinking about the effects of beliefs. Changing the subject–yet another theme that we’re seeing again and again.

What drives student performance (again)?

Leave a reply

I’ve written about this before, but Joe Nocera’s column the other day still bugged me.

Going back to the famous Coleman report in the 1960s, social scientists have contended — and unquestionably proved — that students’ socioeconomic backgrounds vastly outweigh what goes on in the school as factors in determining how much they learn. Richard Rothstein of the Economic Policy Institute lists dozens of reasons why this is so, from the more frequent illness and stress poor students suffer, to the fact that they don’t hear the large vocabularies that middle-class children hear at home.

Number one, that social scientists have contended something since the 1960s is not at all informative. Number two, “unquestionably proving” something is not what social scientists do. We actually don’t prove anything (when dealing with the empirical world, anyway), just disprove alternative explanations. Number three, the sad thing is that–and this is just my impressions–neither social scientists, educators, or the general public really know what makes students learn, or which factors are more important than others.

Number four, what strikes me as a little odd about this debate is that when it comes to bad outcomes, reformers seem to want to emphasize the role of factors outside of the school, like home life. But what about good outcomes? If Nocera is right, doesn’t this mean we shouldn’t credit teachers with helping students learn? That maybe paying them so little is a good thing? I don’t agree with that, but it seems to be a corollary of his claim.

And finally, more causal nihilism.

What needs to be acknowledged, however, is that school reform won’t fix everything. Though some poor students will succeed, others will fail. Demonizing teachers for the failures of poor students, and pretending that reforming the schools is all that is needed, as the reformers tend to do, is both misguided and counterproductive.

Can’t we all just get along? I’m all for admitting the limits of our knowledge, but I really don’t agree with the idea that we can’t get a better sense of what makes students learn–and that we should simultaneously assume we already have it figured out, “unquestionably.”

Why are gas prices so high?

Leave a reply

Ezra Klein beat me to this, but in fact I was prompted by an article in the Boston Globe some days ago. Actually, Klein asks a different question: Why are gas prices rising so much? Here’s his answer, via a UCSD economist.

My first e-mail was to James Hamilton, an economist at the University of California at San Diego who’s looked at gas prices extensively. He explained that you have to break it down into two different components: There’s the price of gasoline, which always rises in the summer, and the price of crude oil, which doesn’t usually rise in the summer. But it’s the price of crude, he said, that’s “the biggest part of the story at the moment.”

The Globe piece is about Obama proposing a commission to investigate the price of gas, basically looking for “illegal activity.” Here’s the key quote from the Justice Department, who are apparently involved:

“Based upon our work and research to date, it is evident that there are regional differences in gasoline prices, as well as differences in the statutory and other legal tools at the government’s disposal. It is also clear that there are lawful reasons for increases in gas prices, given supply and demand,’’ the memo said.

“Nonetheless, where consumers are harmed by unlawful conduct that has the effect of increasing gas prices, state and federal authorities will take swift action,’’ Holder said.

So the bottom line is that there is no mystery here: gas prices are subject to supply and demand, factors that are influenced by what happens outside of the United States. This includes the war in Libya as well as the growth of emerging economies like India and China.

Do high taxes make rich people move away?

Leave a reply

One apparent benefit of the new infusion of blog posts into my formerly print-only RSS is that I’m getting more posts about causal questions and academic studies. There are probably reasons for this, including the different incentives of bloggers and reporters.

Anyhow, I came across an interesting study via Ezra Klein (who found it via economic Robert Frank blogging at the Wall Street Journal). The study asks whether states imposing taxes on millionaires leads the affluent to migrate out of state. The finding was no. I’ll just reprint the portions Klein excerpted:

The study found that the overall population of millionaires increased during the tax period. Some millionaires moved out, of course. But they were more than offset by the creation of new millionaires.

The study dug deeper to figure out whether the millionaires who were moving out did so because of the tax. As a control group, they used New Jersey residents who earned $200,000 to $500,000 — in other words, high-earners who weren’t subject to the tax. They found that the rate of out-migration among millionaires was in line with and rate of out-migration of submillionaires. The tax rate, they concluded, had no measurable impact.

If you follow the links, you’ll see the paper was published in the National Tax Journal. I think this is a really interesting and important study. I have two issues, though. One is the authors’ claim that the new tax constitutes an “experiment.” The other is the use of the lower-income bracket (200 – 500k earners) as a “control group.” On the first point, it is highly unlikely that (a) the imposition of the tax was “random” or (b) the wealthy in New Jersey had no say in whether the tax would be passed. Maybe the wealthy in NJ are just really generous; or, maybe they cut a deal that gave them a big loophole. The failure to randomize creates the possibility of all kinds of stories involving selection.

On the second point, on its face I just have trouble seeing the 200-500k group as controls. Maybe their migration patterns would have been different without the tax, too–they might be expecting to become millionaires rather soon (leading them to move more); or, they might appreciate that those above them are paying more of their fair share (leading them to move less).

Fortunately, there is a lot more data out there to be collected so we can get a better quasi-natural experimental design, as pages 278-279 of the paper suggest.

What is the key to (or cause of) long life?

Leave a reply

“Researchers find conscientiousness might be the key to a long life,” says a summary of a New York Times article I clicked on this morning. Turns out it’s a book review. Now, normally I wouldn’t be so hard on a book review piece, but the book’s authors are described as possessing “statistical findings,” so a higher standard applies.

The first few paragraphs don’t contain any new information beyond the one-sentence summary I quote above. And then we get to the methodology.

In 1990, Dr. Friedman and Leslie Martin, his graduate student at the time, realized that an invaluable resource for studying well-being and longevity existed right in their own state of California. In 1921, Dr. Terman had chosen 1,528 bright San Francisco 11-year-olds for a long-term study of the social predictors of intellectual leadership. Dr. Terman interviewed the children, their families, their teachers. He studied their play habits, their parents’ marriages and their personalities: were they diligent, extroverted, cheerful? He and his team followed up with the participants every five or 10 years. Dr. Terman died in 1956, but colleagues continued the regular interviews with the original subjects, asking the same questions Dr. Terman had asked.

Dr. Friedman and Dr. Martin pored through Dr. Terman’s records, dredged up death certificates and asked Dr. Terman’s questions of study participants’ survivors. They also conducted a group analysis of other similar studies, and collaborated with experts in many fields.

Sounds like a biased sample to me. Strange thing is, the authors seem to recognize this flaw in other studies, as in their characterization of the New England Centenarian Study.

Dr. Friedman pointed out that this was a selected group — the researchers could not study the centenarians themselves, except by self-reporting, so they turned to their children. There was also no control group. The Friedman/Martin/Terman study is unique in that it followed a single set of participants from childhood to death.

I don’t mean to be overly critical of the book–I’m probably just taking issue with the reporters interpretation of it. Indeed, as the closing lines of the piece read,

I have oversimplified, of course, and I, too, would recommend you read the book. It’s a lot more complex than it sounds.

I’m not sure simplification is the real issue, though–it could be the other way round.

Do teachers matter?

Leave a reply

When a teacher moves from a low-income district to a high-income district, their students’ scores go up dramatically. Did the teacher change by moving five miles?

Study after study shows that by far the largest factor explaining low test scores is a child’s family income and neighborhood. If we want to improve student test scores, and much more importantly, student learning, we should guarantee the students’ parents the right to a job at a living wage, and we should be sure kids get adequate food and health care.

From a letter from Dan Clawson in today’s Globe. Interesting claims here. I can’t comment on them, aside from asking for more information. In particular, what studies is Clawson referring to? The rhetoric I remember hearing a lot these days is that what really matters is teachers–but I could be convinced otherwise.

While not solving this debate, the letter does a nice job of putting the public policy issue here in stark terms. If we think student performance is driven by teachers, we need to target the teachers. If we think it’s about class, we need to target redistribution.

Does calcium cause heart attacks and strokes?

Leave a reply

I’m not an epidemiologist, so I don’t know terms like “risk” and “risk factor” (and neither, probably, do most newspaper readers). So when I hear a claim that “calcium increases the risk of heart attack,” I read this as a claim that calcium causes heart attacks.

Which is why I think this study, which I read in the Boston Globe via the Washington Post, is poor quality.

An analysis of data collected about more than 16,000 women who participated in the landmark 2002 Women’s Health Initiative found that those who started taking calcium as part of the study were at increased risk for heart attacks and strokes.

The new analysis from 16,718 women, led by Ian Reid of the University of Auckland, of data published in the British medical journal known as BMJ, found that the women who were not taking calcium when the study started but began taking it when they got into the research project were at 13 to 22 percent increased risk.

So did this subset of women start taking calcium at random? If not, we can stop reading. If so, problems remain. It sounds like at best we have a local average treatment effect–meaning that the studies authors only want to say that going from zero calcium to some indeterminate amount (they don’t say here what the amount is) are at an increased risk. This is “local” because it only applies to the subset of women who took no calcium before.

So practically speaking, this finding is not only of little use, but it’s not really that surprising either. If you suddenly go from zero X to some of X, funny stuff will probably happen.

The sad thing is that stories like this just lead to more fear, confusion, and, ultimately, causal nihilism (p < .10). At least the journal editors included a statement saying that “further studies are needed”–presumably because they don’t think these conclusions are reliable enough. But whether further study–i.e., further research dollars–should go to funding more “studies” like this is not so clear.

Causal nihilism and causal cycling

Leave a reply

Apparently Times columnist Gail Collins recently wrote a column lamenting the conflicting signals sent by the medical community. I didn’t read that column, but I saw the letters to the editor in response. They are pretty interesting–the piece (and a related one in the Week in Review section) seem to have struck a chord. Here’s one letter I liked:

The field of medicine is consumed by pseudoscience, by studies that go in circles and produce contradictory results that benefit only those who want to maintain the confusion that keeps the money rolling in.

Going in circles is something political scientists worry about a lot, but not really in this context. I think it’s true that the studies go in circles–a point beautifully captured by this cartoon. Whether the confusion benefits vested interests is an interesting question that I don’t know the answer to.

Then there is the counter point of view that comes through in these letters, that medicine should adapt as new information becomes available:

The underlying principle of contemporary medicine is that it is an empirical science. When new information is developed, physicians must re-evaluate their opinions in light of it. For ideologues, whose opinions are fixed, this philosophy must seem baffling.

I’m not sure I’d go so far as to say the truth is somewhere in between. What prevents me from saying that is that many medical studies just seem to be poorly done, which would account for the conflicting signals. And it doesn’t seem to be a matter of new methods or new information. Randomization in principle has been known for quite some time, as has the basics of causal reasoning. So I don’t think cycling can be explained away so easily.

So why the word “nihilism” in this post’s title? Another theme of the letters is expressed in the editor’s choice of title for the correspondence: “What to believe?” And it comes through in several of the letters, e.g.

The perception that medicine “on the move” produces confusion, distress and uncertainty is correct, but this reflects an intolerance of uncertainty and illusory confidence in one’s ability to control health through behavior. Diet, exercise, antioxidants and supplements can do only so much. What seems to be more important is one’s genetic makeup.

And this:

Americans have given up on trusting research produced by the health care establishment because contradictory studies are published so frequently, and it’s overwhelming to stay on top of the most up-to-date advice. Too many opinions influenced by drug companies and other special interests have muddied the waters and made it harder to know which studies to trust.

And this:

I blame the relentless advertising of health care products and regimens, some with little proven value, affecting a public that cannot accept disease as something that entails much uncertainty and less control than the health care industry would have us believe.

So the result of all this confusion seems to be people throw up their hands, admit that we can’t control our own bodies and that science is just an illusion. We’re all just pawns in the game played by the big health companies.

What is to be done? I think a basic semester course in statistics and causal inference would be a great start.