How often are scientific studies retracted?
Last week, the coauthor of a widely publicized study from December showing that discussions with gay canvassers could sway voter opinions about gay marriage laws asked the journal that published it to retract it. A flurry of news articles followed, along with questions about the reliability of peer-reviewed research.
There's no comprehensive database of scientific study retractions, but Ivan Oransky's Retraction Watch comes close (and he has a $400,000 grant from the MacArthur Foundation to create one). Orasnky started the blog five years ago with fellow journalist Adam Marcus to help bring transparency to retractions in scientific journals.
He estimates there are at least 500 retractions a year, up more than tenfold since 2001, representing just 0.02 percent of the 2 to 3 million papers published annually. Stunningly, about two-thirds of these retractions are due to misconduct on the part of the researchers, including faking data and plagiarism. That averages out to one retraction a day attributable to dishonest reporting by scientists.
And, Oransky pointed out, that's only the ones who get caught.
"The number of people who are convicted of a crime does not necessarily reflect the number of people who have committed a crime," he told CBS News.
In an op-ed in The New York Times Friday, Oransky and Marcus wrote, "Retractions can be good things, since even scientists often fail to acknowledge their mistakes, preferring instead to allow erroneous findings simply to wither away in the back alleys of unreproducible literature. But they don't surprise those of us who are familiar with how science works; we're surprised only that retractions aren't even more frequent."
The latest controversy began in December, when a political science graduate student at the University of California, Los Angeles, named Michael LaCour published a study in the prestigious journal Science that found that gay canvassers were more effective than straight pollsters in effecting long-term change in voters' opinions on gay marriage.
The findings touched a nerve at a time when same-sex marriage rights are top of mind around the country, and gave hope to activists looking for a meaningful way to change public opinion. Many news organizations, including The New York Times, The Wall Street Journal and NPR, covered the study.
Last week, two researchers hoping to replicate his findings unearthed inconsistencies in LaCour's study that cast doubt on his methods. In light of their investigation, LaCour's coauthor, Columbia University professor Donald P. Green, wrote to Science requesting a retraction.
"Michael LaCour's failure to produce the raw data coupled with the other concerns noted above undermines the credibility of the findings," he wrote. "I am deeply embarrassed by this turn of events and apologize to the editors, reviewers, and readers of Science."
Retraction Watch was the first to break the story Wednesday. The papers and stations that had covered the study when it was published in December rushed after. That day, Science published a so-called "expression of concern," something Oransky said journals do not do lightly.
It said, in part: "Science is urgently working toward the appropriate resolution, while ensuring that a fair process is followed. In the meantime, Science is publishing this Editorial Expression of Concern to alert our readers to the fact that serious questions have been raised about the validity of findings in the LaCour and Green paper."
It can often take years for an expression of concern to turn into a retraction -- if it turns into one at all -- though in this case Oransky predicts the journal won't drag its feet.
"Given all the press, I wouldn't be surprised if they made a decision on it quite quickly," he said. "I can't say what Science would have done if all this were behind closed doors, but in general, if it's raised in public, it's hard to ignore."
And that, perhaps more than anything else, is what makes this potential retraction stand out among all the others. When science meets zeitgeist, things get amplified, often cementing confirmation bias -- the tendency to see proof of one's opinion where there may be none. People are often unwilling or unable to let go of a plausible explanation, even if it's disputed or disproven.
The anti-vaccination movement can be traced back to a single 1998 study linking the measles, mumps and rubella vaccine to autism. That study was retracted 12 years later, but its legacy lives on -- as do measles and mumps.
On the other side of the coin, Oransky said climate change doubters have "seized on (the work he and Marcus do on Retraction Watch) as a good case study why you should question everything about climate change research." Questioning is good, he said, but handful of missing facts don't negate the large body of climate change research that exists.
When a notion takes hold, he said, especially with the proliferation of information on the Internet, "it's difficult to un-ring that bell."
LaCour is expected to release a full response to the allegations against him by Friday.
Speaking about impending retraction decision, Oransky said, "We should be concerned about this and it should force us to think about the incentives that scientists are working under and whether or not peer review is equipped to pick up misconduct and fraud."
He added, "If this story, because the amount of attention it's getting, helps reporters, in particular, to think twice about covering the latest study that sounds a bit too good to be true, the whole kerfuffle -- whatever the downsides of it are -- has accomplished something."