Whenever I read about a study claiming causation using an MRI, the skeptical part of my brain lights up. This piece in the New York Times fashion section is about one such study. One reason I don’t like these studies is they are almost laughably atheoretical–just throw up some MRI plots and talk about this or that “area of the brain.” To wit:
New research suggests that the same areas in the brain that signify physical pain are activated at moments of intense social loss. “When we sat around and thought about the most difficult emotional experiences, we all agreed that it doesn’t get any worse than social rejection,” said the study’s lead author, Ethan F. Kross, an assistant professor of psychology at the University of Michigan.
Then I got to the part about how they conducted the study (which, by the way, the fashion writer gets bonus points for mentioning this at all).
The image of a bunch of social scientists inflicting pain on laboratory volunteers seems creepily Mengelian, but in this case the experiments involved were markedly less cruel. First off, the subjects weren’t socially rejected by the laboratory technicians — each of the 40 volunteers was recruited specifically because he or she felt intensely rejected as a result of a recent (unwanted) breakup.
Oh, ha ha, that’s a relief–the subjects weren’t “rejected” by the researchers.
Um, wait. Selecting on the dependent variable? That is, isn’t there a problem with making inferences about the population from a sample chosen explicitly for its outcome values?
Well, I thought, maybe that’s just how they do things in psychology, and I don’t really know much. Or, maybe this is just the first step in the broader project: identify the correlation in this subsample, then do some random trials.
Turns out, no. Indeed the study seemed to be motivated by a lack of findings from random trials:
Previous research had shown that while social rejection hurt, it did not activate parts of the brain associated with physical distress. But this team found that when the emotional pain was awful enough, those parts of the brain were affected as well, and in equal part. According to the authors, the emotional pain simulated in previous experiments (being told a stranger dislikes them, looking at rejection-themed paintings) wasn’t powerful enough to elicit a true-to-life response. “We were shocked because no prior research had demonstrated this same connection,” Dr. Kross said.
So the moral is, when randomization fails, cook the numbers by selecting a biased sample? Am I being too harsh?